bb-plane-fork/apps/api/plane/app/views/analytic/advance.py
Dheeraj Kumar Ketireddy 514686d9d5
[WEB-4045] feat: restructuring of the external APIs for better maintainability (#7477)
* Basic setup for drf-spectacular

* Updated to only handle /api/v1 endpoints

* feat: add asset and user endpoints with URL routing

- Introduced new asset-related endpoints for user assets and server assets, allowing for asset uploads and management.
- Added user endpoint to retrieve current user information.
- Updated URL routing to include new asset and user patterns.
- Enhanced issue handling with a new search endpoint for issues across multiple fields.
- Expanded member management with a new endpoint for workspace members.

* Group endpoints by tags

* Detailed schema definitions and examples for asset endpoints

* Removed unnecessary extension

* Specify avatar_url field separately

* chore: add project docs

* chore: correct all errors

* chore: added open spec in work items

* feat: enhance cycle API endpoints with detailed OpenAPI specifications

- Updated CycleAPIEndpoint and CycleIssueAPIEndpoint to include detailed OpenAPI schema definitions for GET, POST, PATCH, and DELETE operations.
- Specified allowed HTTP methods for each endpoint in the URL routing.
- Improved documentation for cycle creation, updating, and deletion, including request and response examples.

* chore: added open spec in labels

* chore: work item properties

* feat: enhance API endpoints with OpenAPI specifications and HTTP method definitions

- Added detailed OpenAPI schema definitions for various API endpoints including Intake, Module, and State.
- Specified allowed HTTP methods for each endpoint in the URL routing for better clarity and documentation.
- Improved request and response examples for better understanding of API usage.
- Introduced unarchive functionality for cycles and modules with appropriate endpoint definitions.

* chore: run formatter

* Removed unnecessary settings for authentication

* Refactors OpenAPI documentation structure

Improves the organization and maintainability of the OpenAPI documentation by modularizing the `openapi_spec_helpers.py` file.

The changes include:
- Migrates common parameters, responses, examples, and authentication extensions to separate modules.
- Introduces helper decorators for different endpoint types.
- Updates view imports to use the new module paths.
- Removes the legacy `openapi_spec_helpers.py` file.

This refactoring results in a more structured and easier-to-maintain OpenAPI documentation setup.

* Refactor OpenAPI endpoint specifications

- Removed unnecessary parameters from the OpenAPI documentation for various endpoints in the asset, cycle, and project views.
- Updated request structures to improve clarity and consistency across the API documentation.
- Enhanced response formatting for better readability and maintainability.

* Enhance API documentation with detailed endpoint descriptions

Updated various API endpoints across the application to include comprehensive docstrings that clarify their functionality. Each endpoint now features a summary and detailed description, improving the overall understanding of their purpose and usage. This change enhances the OpenAPI specifications for better developer experience and documentation clarity.

* Enhance API serializers and views with new request structures

- Added new serializers for handling cycle and module issue requests, including `CycleIssueRequestSerializer`, `TransferCycleIssueRequestSerializer`, `ModuleIssueRequestSerializer`, and intake issue creation/updating serializers.
- Updated existing serializers to improve clarity and maintainability, including the `UserAssetUploadSerializer` and `IssueAttachmentUploadSerializer`.
- Refactored API views to utilize the new serializers, enhancing the request handling for cycle and intake issue endpoints.
- Improved OpenAPI documentation by replacing inline request definitions with serializer references for better consistency and readability.

* Refactor OpenAPI documentation and endpoint specifications

- Replaced inline schema definitions with dedicated decorators for various endpoint types, enhancing clarity and maintainability.
- Updated API views to utilize new decorators for user, cycle, intake, module, and project endpoints, improving consistency in OpenAPI documentation.
- Removed unnecessary parameters and responses from endpoint specifications, streamlining the documentation for better readability.
- Enhanced the organization of OpenAPI documentation by modularizing endpoint-specific decorators and parameters.

* chore: correct formatting

* chore: correct formatting for all api folder files

* refactor: clean up serializer imports and test setup

- Removed unused `StateLiteSerializer` import from the serializer module.
- Updated test setup to include a noqa comment for the `django_db_setup` fixture, ensuring clarity in the code.
- Added missing commas in user data dictionary for consistency.

* feat: add project creation and update serializers with validation

- Introduced `ProjectCreateSerializer` and `ProjectUpdateSerializer` to handle project creation and updates, respectively.
- Implemented validation to ensure project leads and default assignees are members of the workspace.
- Updated API views to utilize the new serializers for creating and updating projects, enhancing request handling.
- Added OpenAPI documentation references for the new serializers in the project API endpoints.

* feat: update serializers to include additional read-only fields

* refactor: rename intake issue serializers and enhance structure

- Renamed `CreateIntakeIssueRequestSerializer` to `IntakeIssueCreateSerializer` and `UpdateIntakeIssueRequestSerializer` to `IntakeIssueUpdateSerializer` for clarity.
- Introduced `IssueSerializer` for nested issue data in intake requests, improving the organization of serializer logic.
- Updated API views to utilize the new serializer names, ensuring consistency across the codebase.

* refactor: rename issue serializer for intake and enhance API documentation

- Renamed `IssueSerializer` to `IssueForIntakeSerializer` for better clarity in the context of intake issues.
- Updated references in `IntakeIssueCreateSerializer` and `IntakeIssueUpdateSerializer` to use the new `IssueForIntakeSerializer`.
- Added OpenAPI documentation for the `get_workspace_work_item` endpoint, detailing parameters and responses for improved clarity.

* chore: modules and cycles serializers

* feat: add new serializers for label and issue link management

- Introduced `LabelCreateUpdateSerializer`, `IssueLinkCreateSerializer`, `IssueLinkUpdateSerializer`, and `IssueCommentCreateSerializer` to enhance the handling of label and issue link data.
- Updated existing API views to utilize the new serializers for creating and updating labels, issue links, and comments, improving request handling and validation.
- Added `IssueSearchSerializer` for searching issues, streamlining the search functionality in the API.

* Don't consider read only fields as required

* Add setting to separate request and response definitions

* Fixed avatar_url warning on openapi spec generation

* Made spectacular disabled by default

* Moved spectacular settings into separate file and added detailed descriptions to tags

* Specify methods for asset urls

* Better server names

* Enhance API documentation with summaries for various endpoints

- Added summary descriptions for user asset, cycle, intake, issue, member, module, project, state, and user API endpoints to improve clarity and usability of the API documentation.
- Updated the OpenAPI specifications to reflect these changes, ensuring better understanding for developers interacting with the API.

* Add contact information to OpenAPI settings

- Included contact details for Plane in the OpenAPI settings to enhance API documentation and provide developers with a direct point of contact for support.
- This addition aims to improve the overall usability and accessibility of the API documentation.

* Reordered tags and improved description relavancy

* Enhance OpenAPI documentation for cycle and issue endpoints

- Added response definitions for the `get_cycle_issues` and `delete_cycle_issue` methods in the CycleIssueAPIEndpoint to clarify expected outcomes.
- Included additional response codes for the IssueSearchEndpoint to handle various error scenarios, improving the overall API documentation and usability.

* Enhance serializer documentation across multiple files

- Updated docstrings for various serializers including UserAssetUploadSerializer, AssetUpdateSerializer, and others to provide clearer descriptions of their functionality and usage.
- Improved consistency in formatting and language across serializer classes to enhance readability and maintainability.
- Added detailed explanations for new serializers related to project, module, and cycle management, ensuring comprehensive documentation for developers.

* Refactor API endpoints for cycles, intake, modules, projects, and states

- Replaced existing API endpoint classes with more descriptive names such as CycleListCreateAPIEndpoint, CycleDetailAPIEndpoint, IntakeIssueListCreateAPIEndpoint, and others to enhance clarity.
- Updated URL patterns to reflect the new endpoint names, ensuring consistency across the API.
- Improved documentation and method summaries for better understanding of endpoint functionalities.
- Enhanced query handling in the new endpoint classes to streamline data retrieval and improve performance.

* Refactor issue and label API endpoints for clarity and functionality

- Renamed existing API endpoint classes to more descriptive names such as IssueListCreateAPIEndpoint, IssueDetailAPIEndpoint, LabelListCreateAPIEndpoint, and LabelDetailAPIEndpoint to enhance clarity.
- Updated URL patterns to reflect the new endpoint names, ensuring consistency across the API.
- Improved method summaries and documentation for better understanding of endpoint functionalities.
- Streamlined query handling in the new endpoint classes to enhance data retrieval and performance.

* Refactor asset API endpoint methods and introduce new status enums

- Updated the GenericAssetEndpoint to only allow POST requests for asset creation, removing the GET method.
- Modified the get method to require asset_id, ensuring that asset retrieval is always tied to a specific asset.
- Added new IntakeIssueStatus and ModuleStatus enums to improve clarity and management of asset and module states.
- Enhanced OpenAPI settings to include these new enums for better documentation and usability.

* enforce naming convention

* Added LICENSE to openapi spec

* Enhance OpenAPI documentation for various API endpoints

- Updated API endpoints in asset, cycle, intake, issue, module, project, and state views to include OpenApiRequest and OpenApiExample for better request documentation.
- Added example requests for creating and updating resources, improving clarity for API consumers.
- Ensured consistent use of OpenApi utilities across all relevant endpoints to enhance overall API documentation quality.

* Enhance OpenAPI documentation for various API endpoints

- Added detailed descriptions to multiple API endpoints across asset, cycle, intake, issue, module, project, state, and user views to improve clarity for API consumers.
- Ensured consistent documentation practices by including descriptions that outline the purpose and functionality of each endpoint.
- This update aims to enhance the overall usability and understanding of the API documentation.

* Update OpenAPI examples and enhance project queryset logic

- Changed example fields in OpenAPI documentation for issue comments from "content" to "comment_html" to reflect the correct structure.
- Introduced a new `get_queryset` method in the ProjectDetailAPIEndpoint to filter projects based on user membership and workspace, while also annotating additional project-related data such as total members, cycles, and modules.
- Updated permission checks to use the correct attribute name for project identifiers, ensuring accurate permission handling.

* Enhance OpenAPI documentation and add response examples

- Updated multiple API endpoints across asset, cycle, intake, issue, module, project, state, and user views to include new OpenApiResponse examples for better clarity on expected outcomes.
- Introduced new parameters for project and issue identifiers to improve request handling and documentation consistency.
- Enhanced existing responses with detailed examples to aid API consumers in understanding the expected data structure and error handling.
- This update aims to improve the overall usability and clarity of the API documentation.

* refactor: update terminology from 'issues' to 'work items' across multiple API endpoints for consistency and clarity

* use common timezones from pytz for choices

* Moved the openapi utils to the new folder structure

* Added exception logging in GenericAssetEndpoint to improve error handling

* Fixed code rabbit suggestions

* Refactored IssueDetailAPIEndpoint to streamline issue retrieval and response handling, removing redundant external ID checks and custom ordering logic.

---------

Co-authored-by: pablohashescobar <nikhilschacko@gmail.com>
Co-authored-by: NarayanBavisetti <narayan3119@gmail.com>
2025-07-25 00:17:05 +05:30

364 lines
14 KiB
Python

from rest_framework.response import Response
from rest_framework import status
from typing import Dict, List, Any
from django.db.models import QuerySet, Q, Count
from django.http import HttpRequest
from django.db.models.functions import TruncMonth
from django.utils import timezone
from plane.app.views.base import BaseAPIView
from plane.app.permissions import ROLE, allow_permission
from plane.db.models import (
WorkspaceMember,
Project,
Issue,
Cycle,
Module,
IssueView,
ProjectPage,
Workspace,
ProjectMember,
)
from plane.utils.build_chart import build_analytics_chart
from plane.utils.date_utils import (
get_analytics_filters,
)
class AdvanceAnalyticsBaseView(BaseAPIView):
def initialize_workspace(self, slug: str, type: str) -> None:
self._workspace_slug = slug
self.filters = get_analytics_filters(
slug=slug,
type=type,
user=self.request.user,
date_filter=self.request.GET.get("date_filter", None),
project_ids=self.request.GET.get("project_ids", None),
)
class AdvanceAnalyticsEndpoint(AdvanceAnalyticsBaseView):
def get_filtered_counts(self, queryset: QuerySet) -> Dict[str, int]:
def get_filtered_count() -> int:
if self.filters["analytics_date_range"]:
return queryset.filter(
created_at__gte=self.filters["analytics_date_range"]["current"][
"gte"
],
created_at__lte=self.filters["analytics_date_range"]["current"][
"lte"
],
).count()
return queryset.count()
def get_previous_count() -> int:
if self.filters["analytics_date_range"] and self.filters[
"analytics_date_range"
].get("previous"):
return queryset.filter(
created_at__gte=self.filters["analytics_date_range"]["previous"][
"gte"
],
created_at__lte=self.filters["analytics_date_range"]["previous"][
"lte"
],
).count()
return 0
return {
"count": get_filtered_count(),
# "filter_count": get_previous_count(),
}
def get_overview_data(self) -> Dict[str, Dict[str, int]]:
members_query = WorkspaceMember.objects.filter(
workspace__slug=self._workspace_slug, is_active=True
)
if self.request.GET.get("project_ids", None):
project_ids = self.request.GET.get("project_ids", None)
project_ids = [str(project_id) for project_id in project_ids.split(",")]
members_query = ProjectMember.objects.filter(
project_id__in=project_ids, is_active=True
)
return {
"total_users": self.get_filtered_counts(members_query),
"total_admins": self.get_filtered_counts(
members_query.filter(role=ROLE.ADMIN.value)
),
"total_members": self.get_filtered_counts(
members_query.filter(role=ROLE.MEMBER.value)
),
"total_guests": self.get_filtered_counts(
members_query.filter(role=ROLE.GUEST.value)
),
"total_projects": self.get_filtered_counts(
Project.objects.filter(**self.filters["project_filters"])
),
"total_work_items": self.get_filtered_counts(
Issue.issue_objects.filter(**self.filters["base_filters"])
),
"total_cycles": self.get_filtered_counts(
Cycle.objects.filter(**self.filters["base_filters"])
),
"total_intake": self.get_filtered_counts(
Issue.objects.filter(**self.filters["base_filters"]).filter(
issue_intake__status__in=["-2", "0"]
)
),
}
def get_work_items_stats(self) -> Dict[str, Dict[str, int]]:
base_queryset = Issue.issue_objects.filter(**self.filters["base_filters"])
return {
"total_work_items": self.get_filtered_counts(base_queryset),
"started_work_items": self.get_filtered_counts(
base_queryset.filter(state__group="started")
),
"backlog_work_items": self.get_filtered_counts(
base_queryset.filter(state__group="backlog")
),
"un_started_work_items": self.get_filtered_counts(
base_queryset.filter(state__group="unstarted")
),
"completed_work_items": self.get_filtered_counts(
base_queryset.filter(state__group="completed")
),
}
@allow_permission([ROLE.ADMIN, ROLE.MEMBER], level="WORKSPACE")
def get(self, request: HttpRequest, slug: str) -> Response:
self.initialize_workspace(slug, type="analytics")
tab = request.GET.get("tab", "overview")
if tab == "overview":
return Response(
self.get_overview_data(),
status=status.HTTP_200_OK,
)
elif tab == "work-items":
return Response(
self.get_work_items_stats(),
status=status.HTTP_200_OK,
)
return Response({"message": "Invalid tab"}, status=status.HTTP_400_BAD_REQUEST)
class AdvanceAnalyticsStatsEndpoint(AdvanceAnalyticsBaseView):
def get_project_issues_stats(self) -> QuerySet:
# Get the base queryset with workspace and project filters
base_queryset = Issue.issue_objects.filter(**self.filters["base_filters"])
# Apply date range filter if available
if self.filters["chart_period_range"]:
start_date, end_date = self.filters["chart_period_range"]
base_queryset = base_queryset.filter(
created_at__date__gte=start_date, created_at__date__lte=end_date
)
return (
base_queryset.values("project_id", "project__name")
.annotate(
cancelled_work_items=Count("id", filter=Q(state__group="cancelled")),
completed_work_items=Count("id", filter=Q(state__group="completed")),
backlog_work_items=Count("id", filter=Q(state__group="backlog")),
un_started_work_items=Count("id", filter=Q(state__group="unstarted")),
started_work_items=Count("id", filter=Q(state__group="started")),
)
.order_by("project_id")
)
def get_work_items_stats(self) -> Dict[str, Dict[str, int]]:
base_queryset = Issue.issue_objects.filter(**self.filters["base_filters"])
return (
base_queryset.values("project_id", "project__name")
.annotate(
cancelled_work_items=Count("id", filter=Q(state__group="cancelled")),
completed_work_items=Count("id", filter=Q(state__group="completed")),
backlog_work_items=Count("id", filter=Q(state__group="backlog")),
un_started_work_items=Count("id", filter=Q(state__group="unstarted")),
started_work_items=Count("id", filter=Q(state__group="started")),
)
.order_by("project_id")
)
@allow_permission([ROLE.ADMIN, ROLE.MEMBER], level="WORKSPACE")
def get(self, request: HttpRequest, slug: str) -> Response:
self.initialize_workspace(slug, type="chart")
type = request.GET.get("type", "work-items")
if type == "work-items":
return Response(
self.get_work_items_stats(),
status=status.HTTP_200_OK,
)
return Response({"message": "Invalid type"}, status=status.HTTP_400_BAD_REQUEST)
class AdvanceAnalyticsChartEndpoint(AdvanceAnalyticsBaseView):
def project_chart(self) -> List[Dict[str, Any]]:
# Get the base queryset with workspace and project filters
base_queryset = Issue.issue_objects.filter(**self.filters["base_filters"])
date_filter = {}
# Apply date range filter if available
if self.filters["chart_period_range"]:
start_date, end_date = self.filters["chart_period_range"]
date_filter = {
"created_at__date__gte": start_date,
"created_at__date__lte": end_date,
}
total_work_items = base_queryset.filter(**date_filter).count()
total_cycles = Cycle.objects.filter(
**self.filters["base_filters"], **date_filter
).count()
total_modules = Module.objects.filter(
**self.filters["base_filters"], **date_filter
).count()
total_intake = Issue.objects.filter(
issue_intake__isnull=False, **self.filters["base_filters"], **date_filter
).count()
total_members = WorkspaceMember.objects.filter(
workspace__slug=self._workspace_slug, is_active=True, **date_filter
).count()
total_pages = ProjectPage.objects.filter(
**self.filters["base_filters"], **date_filter
).count()
total_views = IssueView.objects.filter(
**self.filters["base_filters"], **date_filter
).count()
data = {
"work_items": total_work_items,
"cycles": total_cycles,
"modules": total_modules,
"intake": total_intake,
"members": total_members,
"pages": total_pages,
"views": total_views,
}
return [
{
"key": key,
"name": key.replace("_", " ").title(),
"count": value or 0,
}
for key, value in data.items()
]
def work_item_completion_chart(self) -> Dict[str, Any]:
# Get the base queryset
queryset = (
Issue.issue_objects.filter(**self.filters["base_filters"])
.select_related("workspace", "state", "parent")
.prefetch_related(
"assignees", "labels", "issue_module__module", "issue_cycle__cycle"
)
)
workspace = Workspace.objects.get(slug=self._workspace_slug)
start_date = workspace.created_at.date().replace(day=1)
# Apply date range filter if available
if self.filters["chart_period_range"]:
start_date, end_date = self.filters["chart_period_range"]
queryset = queryset.filter(
created_at__date__gte=start_date, created_at__date__lte=end_date
)
# Annotate by month and count
monthly_stats = (
queryset.annotate(month=TruncMonth("created_at"))
.values("month")
.annotate(
created_count=Count("id"),
completed_count=Count("id", filter=Q(state__group="completed")),
)
.order_by("month")
)
# Create dictionary of month -> counts
stats_dict = {
stat["month"].strftime("%Y-%m-%d"): {
"created_count": stat["created_count"],
"completed_count": stat["completed_count"],
}
for stat in monthly_stats
}
# Generate monthly data (ensure months with 0 count are included)
data = []
# include the current date at the end
end_date = timezone.now().date()
last_month = end_date.replace(day=1)
current_month = start_date
while current_month <= last_month:
date_str = current_month.strftime("%Y-%m-%d")
stats = stats_dict.get(date_str, {"created_count": 0, "completed_count": 0})
data.append(
{
"key": date_str,
"name": date_str,
"count": stats["created_count"],
"completed_issues": stats["completed_count"],
"created_issues": stats["created_count"],
}
)
# Move to next month
if current_month.month == 12:
current_month = current_month.replace(
year=current_month.year + 1, month=1
)
else:
current_month = current_month.replace(month=current_month.month + 1)
schema = {
"completed_issues": "completed_issues",
"created_issues": "created_issues",
}
return {"data": data, "schema": schema}
@allow_permission([ROLE.ADMIN, ROLE.MEMBER], level="WORKSPACE")
def get(self, request: HttpRequest, slug: str) -> Response:
self.initialize_workspace(slug, type="chart")
type = request.GET.get("type", "projects")
group_by = request.GET.get("group_by", None)
x_axis = request.GET.get("x_axis", "PRIORITY")
if type == "projects":
return Response(self.project_chart(), status=status.HTTP_200_OK)
elif type == "custom-work-items":
queryset = (
Issue.issue_objects.filter(**self.filters["base_filters"])
.select_related("workspace", "state", "parent")
.prefetch_related(
"assignees", "labels", "issue_module__module", "issue_cycle__cycle"
)
)
# Apply date range filter if available
if self.filters["chart_period_range"]:
start_date, end_date = self.filters["chart_period_range"]
queryset = queryset.filter(
created_at__date__gte=start_date, created_at__date__lte=end_date
)
return Response(
build_analytics_chart(queryset, x_axis, group_by),
status=status.HTTP_200_OK,
)
elif type == "work-items":
return Response(
self.work_item_completion_chart(),
status=status.HTTP_200_OK,
)
return Response({"message": "Invalid type"}, status=status.HTTP_400_BAD_REQUEST)