gitea/routers/web/repo/issue.go

683 lines
19 KiB
Go
Raw Normal View History

2014-03-23 01:50:50 +08:00
// Copyright 2014 The Gogs Authors. All rights reserved.
// Copyright 2018 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
2014-03-23 01:50:50 +08:00
package repo
import (
"errors"
"fmt"
"html/template"
"net/http"
"net/url"
2017-03-15 09:10:35 +08:00
"strconv"
"strings"
"code.gitea.io/gitea/models/db"
issues_model "code.gitea.io/gitea/models/issues"
"code.gitea.io/gitea/models/organization"
access_model "code.gitea.io/gitea/models/perm/access"
project_model "code.gitea.io/gitea/models/project"
"code.gitea.io/gitea/models/renderhelper"
repo_model "code.gitea.io/gitea/models/repo"
"code.gitea.io/gitea/models/unit"
user_model "code.gitea.io/gitea/models/user"
"code.gitea.io/gitea/modules/base"
"code.gitea.io/gitea/modules/log"
"code.gitea.io/gitea/modules/markup/markdown"
"code.gitea.io/gitea/modules/optional"
api "code.gitea.io/gitea/modules/structs"
"code.gitea.io/gitea/modules/templates"
2017-01-25 10:43:02 +08:00
"code.gitea.io/gitea/modules/util"
Move macaron to chi (#14293) Use [chi](https://github.com/go-chi/chi) instead of the forked [macaron](https://gitea.com/macaron/macaron). Since macaron and chi have conflicts with session share, this big PR becomes a have-to thing. According my previous idea, we can replace macaron step by step but I'm wrong. :( Below is a list of big changes on this PR. - [x] Define `context.ResponseWriter` interface with an implementation `context.Response`. - [x] Use chi instead of macaron, and also a customize `Route` to wrap chi so that the router usage is similar as before. - [x] Create different routers for `web`, `api`, `internal` and `install` so that the codes will be more clear and no magic . - [x] Use https://github.com/unrolled/render instead of macaron's internal render - [x] Use https://github.com/NYTimes/gziphandler instead of https://gitea.com/macaron/gzip - [x] Use https://gitea.com/go-chi/session which is a modified version of https://gitea.com/macaron/session and removed `nodb` support since it will not be maintained. **BREAK** - [x] Use https://gitea.com/go-chi/captcha which is a modified version of https://gitea.com/macaron/captcha - [x] Use https://gitea.com/go-chi/cache which is a modified version of https://gitea.com/macaron/cache - [x] Use https://gitea.com/go-chi/binding which is a modified version of https://gitea.com/macaron/binding - [x] Use https://github.com/go-chi/cors instead of https://gitea.com/macaron/cors - [x] Dropped https://gitea.com/macaron/i18n and make a new one in `code.gitea.io/gitea/modules/translation` - [x] Move validation form structs from `code.gitea.io/gitea/modules/auth` to `code.gitea.io/gitea/modules/forms` to avoid dependency cycle. - [x] Removed macaron log service because it's not need any more. **BREAK** - [x] All form structs have to be get by `web.GetForm(ctx)` in the route function but not as a function parameter on routes definition. - [x] Move Git HTTP protocol implementation to use routers directly. - [x] Fix the problem that chi routes don't support trailing slash but macaron did. - [x] `/api/v1/swagger` now will be redirect to `/api/swagger` but not render directly so that `APIContext` will not create a html render. Notices: - Chi router don't support request with trailing slash - Integration test `TestUserHeatmap` maybe mysql version related. It's failed on my macOS(mysql 5.7.29 installed via brew) but succeed on CI. Co-authored-by: 6543 <6543@obermui.de>
2021-01-26 23:36:53 +08:00
"code.gitea.io/gitea/modules/web"
"code.gitea.io/gitea/routers/common"
"code.gitea.io/gitea/services/context"
"code.gitea.io/gitea/services/convert"
"code.gitea.io/gitea/services/forms"
issue_service "code.gitea.io/gitea/services/issue"
)
const (
tplAttachment base.TplName = "repo/issue/view_content/attachments"
tplIssues base.TplName = "repo/issue/list"
tplIssueNew base.TplName = "repo/issue/new"
tplIssueChoose base.TplName = "repo/issue/choose"
tplIssueView base.TplName = "repo/issue/view"
tplReactions base.TplName = "repo/issue/view_content/reactions"
issueTemplateKey = "IssueTemplate"
issueTemplateTitleKey = "IssueTemplateTitle"
)
// IssueTemplateCandidates issue templates
var IssueTemplateCandidates = []string{
"ISSUE_TEMPLATE.md",
Support Issue forms and PR forms (#20987) * feat: extend issue template for yaml * feat: support yaml template * feat: render form to markdown * feat: support yaml template for pr * chore: rename to Fields * feat: template unmarshal * feat: split template * feat: render to markdown * feat: use full name as template file name * chore: remove useless file * feat: use dropdown of fomantic ui * feat: update input style * docs: more comments * fix: render text without render * chore: fix lint error * fix: support use description as about in markdown * fix: add field class in form * chore: generate swagger * feat: validate template * feat: support is_nummber and regex * test: fix broken unit tests * fix: ignore empty body of md template * fix: make multiple easymde editors work in one page * feat: better UI * fix: js error in pr form * chore: generate swagger * feat: support regex validation * chore: generate swagger * fix: refresh each markdown editor * chore: give up required validation * fix: correct issue template candidates * fix: correct checkboxes style * chore: ignore .hugo_build.lock in docs * docs: separate out a new doc for merge templates * docs: introduce syntax of yaml template * feat: show a alert for invalid templates * test: add case for a valid template * fix: correct attributes of required checkbox * fix: add class not-under-easymde for dropzone * fix: use more back-quotes * chore: remove translation in zh-CN * fix EasyMDE statusbar margin * fix: remove repeated blocks * fix: reuse regex for quotes Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-09-02 15:58:49 +08:00
"ISSUE_TEMPLATE.yaml",
"ISSUE_TEMPLATE.yml",
"issue_template.md",
Support Issue forms and PR forms (#20987) * feat: extend issue template for yaml * feat: support yaml template * feat: render form to markdown * feat: support yaml template for pr * chore: rename to Fields * feat: template unmarshal * feat: split template * feat: render to markdown * feat: use full name as template file name * chore: remove useless file * feat: use dropdown of fomantic ui * feat: update input style * docs: more comments * fix: render text without render * chore: fix lint error * fix: support use description as about in markdown * fix: add field class in form * chore: generate swagger * feat: validate template * feat: support is_nummber and regex * test: fix broken unit tests * fix: ignore empty body of md template * fix: make multiple easymde editors work in one page * feat: better UI * fix: js error in pr form * chore: generate swagger * feat: support regex validation * chore: generate swagger * fix: refresh each markdown editor * chore: give up required validation * fix: correct issue template candidates * fix: correct checkboxes style * chore: ignore .hugo_build.lock in docs * docs: separate out a new doc for merge templates * docs: introduce syntax of yaml template * feat: show a alert for invalid templates * test: add case for a valid template * fix: correct attributes of required checkbox * fix: add class not-under-easymde for dropzone * fix: use more back-quotes * chore: remove translation in zh-CN * fix EasyMDE statusbar margin * fix: remove repeated blocks * fix: reuse regex for quotes Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-09-02 15:58:49 +08:00
"issue_template.yaml",
"issue_template.yml",
".gitea/ISSUE_TEMPLATE.md",
Support Issue forms and PR forms (#20987) * feat: extend issue template for yaml * feat: support yaml template * feat: render form to markdown * feat: support yaml template for pr * chore: rename to Fields * feat: template unmarshal * feat: split template * feat: render to markdown * feat: use full name as template file name * chore: remove useless file * feat: use dropdown of fomantic ui * feat: update input style * docs: more comments * fix: render text without render * chore: fix lint error * fix: support use description as about in markdown * fix: add field class in form * chore: generate swagger * feat: validate template * feat: support is_nummber and regex * test: fix broken unit tests * fix: ignore empty body of md template * fix: make multiple easymde editors work in one page * feat: better UI * fix: js error in pr form * chore: generate swagger * feat: support regex validation * chore: generate swagger * fix: refresh each markdown editor * chore: give up required validation * fix: correct issue template candidates * fix: correct checkboxes style * chore: ignore .hugo_build.lock in docs * docs: separate out a new doc for merge templates * docs: introduce syntax of yaml template * feat: show a alert for invalid templates * test: add case for a valid template * fix: correct attributes of required checkbox * fix: add class not-under-easymde for dropzone * fix: use more back-quotes * chore: remove translation in zh-CN * fix EasyMDE statusbar margin * fix: remove repeated blocks * fix: reuse regex for quotes Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-09-02 15:58:49 +08:00
".gitea/ISSUE_TEMPLATE.yaml",
".gitea/ISSUE_TEMPLATE.yml",
".gitea/issue_template.md",
".gitea/issue_template.yaml",
".gitea/issue_template.yml",
".github/ISSUE_TEMPLATE.md",
Support Issue forms and PR forms (#20987) * feat: extend issue template for yaml * feat: support yaml template * feat: render form to markdown * feat: support yaml template for pr * chore: rename to Fields * feat: template unmarshal * feat: split template * feat: render to markdown * feat: use full name as template file name * chore: remove useless file * feat: use dropdown of fomantic ui * feat: update input style * docs: more comments * fix: render text without render * chore: fix lint error * fix: support use description as about in markdown * fix: add field class in form * chore: generate swagger * feat: validate template * feat: support is_nummber and regex * test: fix broken unit tests * fix: ignore empty body of md template * fix: make multiple easymde editors work in one page * feat: better UI * fix: js error in pr form * chore: generate swagger * feat: support regex validation * chore: generate swagger * fix: refresh each markdown editor * chore: give up required validation * fix: correct issue template candidates * fix: correct checkboxes style * chore: ignore .hugo_build.lock in docs * docs: separate out a new doc for merge templates * docs: introduce syntax of yaml template * feat: show a alert for invalid templates * test: add case for a valid template * fix: correct attributes of required checkbox * fix: add class not-under-easymde for dropzone * fix: use more back-quotes * chore: remove translation in zh-CN * fix EasyMDE statusbar margin * fix: remove repeated blocks * fix: reuse regex for quotes Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-09-02 15:58:49 +08:00
".github/ISSUE_TEMPLATE.yaml",
".github/ISSUE_TEMPLATE.yml",
".github/issue_template.md",
Support Issue forms and PR forms (#20987) * feat: extend issue template for yaml * feat: support yaml template * feat: render form to markdown * feat: support yaml template for pr * chore: rename to Fields * feat: template unmarshal * feat: split template * feat: render to markdown * feat: use full name as template file name * chore: remove useless file * feat: use dropdown of fomantic ui * feat: update input style * docs: more comments * fix: render text without render * chore: fix lint error * fix: support use description as about in markdown * fix: add field class in form * chore: generate swagger * feat: validate template * feat: support is_nummber and regex * test: fix broken unit tests * fix: ignore empty body of md template * fix: make multiple easymde editors work in one page * feat: better UI * fix: js error in pr form * chore: generate swagger * feat: support regex validation * chore: generate swagger * fix: refresh each markdown editor * chore: give up required validation * fix: correct issue template candidates * fix: correct checkboxes style * chore: ignore .hugo_build.lock in docs * docs: separate out a new doc for merge templates * docs: introduce syntax of yaml template * feat: show a alert for invalid templates * test: add case for a valid template * fix: correct attributes of required checkbox * fix: add class not-under-easymde for dropzone * fix: use more back-quotes * chore: remove translation in zh-CN * fix EasyMDE statusbar margin * fix: remove repeated blocks * fix: reuse regex for quotes Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-09-02 15:58:49 +08:00
".github/issue_template.yaml",
".github/issue_template.yml",
}
// MustAllowUserComment checks to make sure if an issue is locked.
// If locked and user has permissions to write to the repository,
// then the comment is allowed, else it is blocked
func MustAllowUserComment(ctx *context.Context) {
issue := GetActionIssue(ctx)
if ctx.Written() {
return
}
if issue.IsLocked && !ctx.Repo.CanWriteIssuesOrPulls(issue.IsPull) && !ctx.Doer.IsAdmin {
ctx.Flash.Error(ctx.Tr("repo.issues.comment_on_locked"))
ctx.Redirect(issue.Link())
return
}
}
2016-11-24 15:04:31 +08:00
// MustEnableIssues check if repository enable internal issues
2016-03-12 00:56:52 +08:00
func MustEnableIssues(ctx *context.Context) {
if !ctx.Repo.CanRead(unit.TypeIssues) &&
!ctx.Repo.CanRead(unit.TypeExternalTracker) {
ctx.NotFound("MustEnableIssues", nil)
2016-03-07 12:57:46 +08:00
return
2015-12-05 10:30:33 +08:00
}
unit, err := ctx.Repo.Repository.GetUnit(ctx, unit.TypeExternalTracker)
if err == nil {
ctx.Redirect(unit.ExternalTrackerConfig().ExternalTrackerURL)
return
}
2015-12-05 10:30:33 +08:00
}
// MustAllowPulls check if repository enable pull requests and user have right to do that
2016-03-12 00:56:52 +08:00
func MustAllowPulls(ctx *context.Context) {
if !ctx.Repo.Repository.CanEnablePulls() || !ctx.Repo.CanRead(unit.TypePullRequests) {
ctx.NotFound("MustAllowPulls", nil)
2016-03-07 12:57:46 +08:00
return
2015-12-05 10:30:33 +08:00
}
2015-12-20 11:07:06 +08:00
2016-03-07 12:57:46 +08:00
// User can send pull request if owns a forked repository.
if ctx.IsSigned && repo_model.HasForkedRepo(ctx, ctx.Doer.ID, ctx.Repo.Repository.ID) {
2016-03-07 12:57:46 +08:00
ctx.Repo.PullRequest.Allowed = true
ctx.Repo.PullRequest.HeadInfoSubURL = url.PathEscape(ctx.Doer.Name) + ":" + util.PathEscapeSegments(ctx.Repo.BranchName)
2016-03-07 12:57:46 +08:00
}
2015-12-05 10:30:33 +08:00
}
func retrieveProjectsInternal(ctx *context.Context, repo *repo_model.Repository) (open, closed []*project_model.Project) {
// Distinguish whether the owner of the repository
// is an individual or an organization
repoOwnerType := project_model.TypeIndividual
if repo.Owner.IsOrganization() {
repoOwnerType = project_model.TypeOrganization
}
projectsUnit := repo.MustGetUnit(ctx, unit.TypeProjects)
var openProjects []*project_model.Project
var closedProjects []*project_model.Project
var err error
if projectsUnit.ProjectsConfig().IsProjectsAllowed(repo_model.ProjectsModeRepo) {
openProjects, err = db.Find[project_model.Project](ctx, project_model.SearchOptions{
ListOptions: db.ListOptionsAll,
RepoID: repo.ID,
IsClosed: optional.Some(false),
Type: project_model.TypeRepository,
})
if err != nil {
ctx.ServerError("GetProjects", err)
return nil, nil
}
closedProjects, err = db.Find[project_model.Project](ctx, project_model.SearchOptions{
ListOptions: db.ListOptionsAll,
RepoID: repo.ID,
IsClosed: optional.Some(true),
Type: project_model.TypeRepository,
})
if err != nil {
ctx.ServerError("GetProjects", err)
return nil, nil
}
}
if projectsUnit.ProjectsConfig().IsProjectsAllowed(repo_model.ProjectsModeOwner) {
openProjects2, err := db.Find[project_model.Project](ctx, project_model.SearchOptions{
ListOptions: db.ListOptionsAll,
OwnerID: repo.OwnerID,
IsClosed: optional.Some(false),
Type: repoOwnerType,
})
if err != nil {
ctx.ServerError("GetProjects", err)
return nil, nil
}
openProjects = append(openProjects, openProjects2...)
closedProjects2, err := db.Find[project_model.Project](ctx, project_model.SearchOptions{
ListOptions: db.ListOptionsAll,
OwnerID: repo.OwnerID,
IsClosed: optional.Some(true),
Type: repoOwnerType,
})
if err != nil {
ctx.ServerError("GetProjects", err)
return nil, nil
}
closedProjects = append(closedProjects, closedProjects2...)
}
return openProjects, closedProjects
}
// GetActionIssue will return the issue which is used in the context.
func GetActionIssue(ctx *context.Context) *issues_model.Issue {
issue, err := issues_model.GetIssueByIndex(ctx, ctx.Repo.Repository.ID, ctx.PathParamInt64(":index"))
if err != nil {
ctx.NotFoundOrServerError("GetIssueByIndex", issues_model.IsErrIssueNotExist, err)
return nil
}
issue.Repo = ctx.Repo.Repository
checkIssueRights(ctx, issue)
if ctx.Written() {
return nil
}
if err = issue.LoadAttributes(ctx); err != nil {
ctx.ServerError("LoadAttributes", err)
return nil
}
return issue
}
func checkIssueRights(ctx *context.Context, issue *issues_model.Issue) {
if issue.IsPull && !ctx.Repo.CanRead(unit.TypePullRequests) ||
!issue.IsPull && !ctx.Repo.CanRead(unit.TypeIssues) {
ctx.NotFound("IssueOrPullRequestUnitNotAllowed", nil)
}
}
func getActionIssues(ctx *context.Context) issues_model.IssueList {
commaSeparatedIssueIDs := ctx.FormString("issue_ids")
if len(commaSeparatedIssueIDs) == 0 {
return nil
}
issueIDs := make([]int64, 0, 10)
for _, stringIssueID := range strings.Split(commaSeparatedIssueIDs, ",") {
issueID, err := strconv.ParseInt(stringIssueID, 10, 64)
if err != nil {
ctx.ServerError("ParseInt", err)
return nil
}
issueIDs = append(issueIDs, issueID)
}
issues, err := issues_model.GetIssuesByIDs(ctx, issueIDs)
if err != nil {
ctx.ServerError("GetIssuesByIDs", err)
return nil
Refactor and enhance issue indexer to support both searching, filtering and paging (#26012) Fix #24662. Replace #24822 and #25708 (although it has been merged) ## Background In the past, Gitea supported issue searching with a keyword and conditions in a less efficient way. It worked by searching for issues with the keyword and obtaining limited IDs (as it is heavy to get all) on the indexer (bleve/elasticsearch/meilisearch), and then querying with conditions on the database to find a subset of the found IDs. This is why the results could be incomplete. To solve this issue, we need to store all fields that could be used as conditions in the indexer and support both keyword and additional conditions when searching with the indexer. ## Major changes - Redefine `IndexerData` to include all fields that could be used as filter conditions. - Refactor `Search(ctx context.Context, kw string, repoIDs []int64, limit, start int, state string)` to `Search(ctx context.Context, options *SearchOptions)`, so it supports more conditions now. - Change the data type stored in `issueIndexerQueue`. Use `IndexerMetadata` instead of `IndexerData` in case the data has been updated while it is in the queue. This also reduces the storage size of the queue. - Enhance searching with Bleve/Elasticsearch/Meilisearch, make them fully support `SearchOptions`. Also, update the data versions. - Keep most logic of database indexer, but remove `issues.SearchIssueIDsByKeyword` in `models` to avoid confusion where is the entry point to search issues. - Start a Meilisearch instance to test it in unit tests. - Add unit tests with almost full coverage to test Bleve/Elasticsearch/Meilisearch indexer. --------- Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-07-31 14:28:53 +08:00
}
// Check access rights for all issues
issueUnitEnabled := ctx.Repo.CanRead(unit.TypeIssues)
prUnitEnabled := ctx.Repo.CanRead(unit.TypePullRequests)
for _, issue := range issues {
if issue.RepoID != ctx.Repo.Repository.ID {
ctx.NotFound("some issue's RepoID is incorrect", errors.New("some issue's RepoID is incorrect"))
return nil
}
if issue.IsPull && !prUnitEnabled || !issue.IsPull && !issueUnitEnabled {
ctx.NotFound("IssueOrPullRequestUnitNotAllowed", nil)
return nil
}
if err = issue.LoadAttributes(ctx); err != nil {
ctx.ServerError("LoadAttributes", err)
return nil
}
Refactor and enhance issue indexer to support both searching, filtering and paging (#26012) Fix #24662. Replace #24822 and #25708 (although it has been merged) ## Background In the past, Gitea supported issue searching with a keyword and conditions in a less efficient way. It worked by searching for issues with the keyword and obtaining limited IDs (as it is heavy to get all) on the indexer (bleve/elasticsearch/meilisearch), and then querying with conditions on the database to find a subset of the found IDs. This is why the results could be incomplete. To solve this issue, we need to store all fields that could be used as conditions in the indexer and support both keyword and additional conditions when searching with the indexer. ## Major changes - Redefine `IndexerData` to include all fields that could be used as filter conditions. - Refactor `Search(ctx context.Context, kw string, repoIDs []int64, limit, start int, state string)` to `Search(ctx context.Context, options *SearchOptions)`, so it supports more conditions now. - Change the data type stored in `issueIndexerQueue`. Use `IndexerMetadata` instead of `IndexerData` in case the data has been updated while it is in the queue. This also reduces the storage size of the queue. - Enhance searching with Bleve/Elasticsearch/Meilisearch, make them fully support `SearchOptions`. Also, update the data versions. - Keep most logic of database indexer, but remove `issues.SearchIssueIDsByKeyword` in `models` to avoid confusion where is the entry point to search issues. - Start a Meilisearch instance to test it in unit tests. - Add unit tests with almost full coverage to test Bleve/Elasticsearch/Meilisearch indexer. --------- Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-07-31 14:28:53 +08:00
}
return issues
}
// GetIssueInfo get an issue of a repository
func GetIssueInfo(ctx *context.Context) {
issue, err := issues_model.GetIssueWithAttrsByIndex(ctx, ctx.Repo.Repository.ID, ctx.PathParamInt64(":index"))
if err != nil {
if issues_model.IsErrIssueNotExist(err) {
ctx.Error(http.StatusNotFound)
} else {
ctx.Error(http.StatusInternalServerError, "GetIssueByIndex", err.Error())
}
return
}
if issue.IsPull {
// Need to check if Pulls are enabled and we can read Pulls
if !ctx.Repo.Repository.CanEnablePulls() || !ctx.Repo.CanRead(unit.TypePullRequests) {
ctx.Error(http.StatusNotFound)
return
}
Refactor and enhance issue indexer to support both searching, filtering and paging (#26012) Fix #24662. Replace #24822 and #25708 (although it has been merged) ## Background In the past, Gitea supported issue searching with a keyword and conditions in a less efficient way. It worked by searching for issues with the keyword and obtaining limited IDs (as it is heavy to get all) on the indexer (bleve/elasticsearch/meilisearch), and then querying with conditions on the database to find a subset of the found IDs. This is why the results could be incomplete. To solve this issue, we need to store all fields that could be used as conditions in the indexer and support both keyword and additional conditions when searching with the indexer. ## Major changes - Redefine `IndexerData` to include all fields that could be used as filter conditions. - Refactor `Search(ctx context.Context, kw string, repoIDs []int64, limit, start int, state string)` to `Search(ctx context.Context, options *SearchOptions)`, so it supports more conditions now. - Change the data type stored in `issueIndexerQueue`. Use `IndexerMetadata` instead of `IndexerData` in case the data has been updated while it is in the queue. This also reduces the storage size of the queue. - Enhance searching with Bleve/Elasticsearch/Meilisearch, make them fully support `SearchOptions`. Also, update the data versions. - Keep most logic of database indexer, but remove `issues.SearchIssueIDsByKeyword` in `models` to avoid confusion where is the entry point to search issues. - Start a Meilisearch instance to test it in unit tests. - Add unit tests with almost full coverage to test Bleve/Elasticsearch/Meilisearch indexer. --------- Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-07-31 14:28:53 +08:00
} else {
// Need to check if Issues are enabled and we can read Issues
if !ctx.Repo.CanRead(unit.TypeIssues) {
ctx.Error(http.StatusNotFound)
return
}
Refactor and enhance issue indexer to support both searching, filtering and paging (#26012) Fix #24662. Replace #24822 and #25708 (although it has been merged) ## Background In the past, Gitea supported issue searching with a keyword and conditions in a less efficient way. It worked by searching for issues with the keyword and obtaining limited IDs (as it is heavy to get all) on the indexer (bleve/elasticsearch/meilisearch), and then querying with conditions on the database to find a subset of the found IDs. This is why the results could be incomplete. To solve this issue, we need to store all fields that could be used as conditions in the indexer and support both keyword and additional conditions when searching with the indexer. ## Major changes - Redefine `IndexerData` to include all fields that could be used as filter conditions. - Refactor `Search(ctx context.Context, kw string, repoIDs []int64, limit, start int, state string)` to `Search(ctx context.Context, options *SearchOptions)`, so it supports more conditions now. - Change the data type stored in `issueIndexerQueue`. Use `IndexerMetadata` instead of `IndexerData` in case the data has been updated while it is in the queue. This also reduces the storage size of the queue. - Enhance searching with Bleve/Elasticsearch/Meilisearch, make them fully support `SearchOptions`. Also, update the data versions. - Keep most logic of database indexer, but remove `issues.SearchIssueIDsByKeyword` in `models` to avoid confusion where is the entry point to search issues. - Start a Meilisearch instance to test it in unit tests. - Add unit tests with almost full coverage to test Bleve/Elasticsearch/Meilisearch indexer. --------- Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-07-31 14:28:53 +08:00
}
ctx.JSON(http.StatusOK, map[string]any{
"convertedIssue": convert.ToIssue(ctx, ctx.Doer, issue),
"renderedLabels": templates.NewRenderUtils(ctx).RenderLabels(issue.Labels, ctx.Repo.RepoLink, issue),
})
}
// UpdateIssueTitle change issue's title
func UpdateIssueTitle(ctx *context.Context) {
issue := GetActionIssue(ctx)
if ctx.Written() {
return
}
if !ctx.IsSigned || (!issue.IsPoster(ctx.Doer.ID) && !ctx.Repo.CanWriteIssuesOrPulls(issue.IsPull)) {
ctx.Error(http.StatusForbidden)
return
}
title := ctx.FormTrim("title")
if len(title) == 0 {
ctx.Error(http.StatusNoContent)
2017-03-15 09:10:35 +08:00
return
}
if err := issue_service.ChangeTitle(ctx, issue, ctx.Doer, title); err != nil {
ctx.ServerError("ChangeTitle", err)
return
}
ctx.JSON(http.StatusOK, map[string]any{
"title": issue.Title,
})
}
// UpdateIssueRef change issue's ref (branch)
func UpdateIssueRef(ctx *context.Context) {
issue := GetActionIssue(ctx)
if ctx.Written() {
return
}
if !ctx.IsSigned || (!issue.IsPoster(ctx.Doer.ID) && !ctx.Repo.CanWriteIssuesOrPulls(issue.IsPull)) || issue.IsPull {
ctx.Error(http.StatusForbidden)
return
}
ref := ctx.FormTrim("ref")
if err := issue_service.ChangeIssueRef(ctx, issue, ctx.Doer, ref); err != nil {
ctx.ServerError("ChangeRef", err)
return
}
ctx.JSON(http.StatusOK, map[string]any{
"ref": ref,
})
}
// UpdateIssueContent change issue's content
func UpdateIssueContent(ctx *context.Context) {
issue := GetActionIssue(ctx)
if ctx.Written() {
return
}
if !ctx.IsSigned || (ctx.Doer.ID != issue.PosterID && !ctx.Repo.CanWriteIssuesOrPulls(issue.IsPull)) {
ctx.Error(http.StatusForbidden)
2015-08-20 04:31:28 +08:00
return
}
if err := issue_service.ChangeContent(ctx, issue, ctx.Doer, ctx.Req.FormValue("content"), ctx.FormInt("content_version")); err != nil {
if errors.Is(err, user_model.ErrBlockedUser) {
ctx.JSONError(ctx.Tr("repo.issues.edit.blocked_user"))
} else if errors.Is(err, issues_model.ErrIssueAlreadyChanged) {
if issue.IsPull {
ctx.JSONError(ctx.Tr("repo.pulls.edit.already_changed"))
} else {
ctx.JSONError(ctx.Tr("repo.issues.edit.already_changed"))
}
} else {
ctx.ServerError("ChangeContent", err)
}
2015-08-20 04:31:28 +08:00
return
}
// when update the request doesn't intend to update attachments (eg: change checkbox state), ignore attachment updates
if !ctx.FormBool("ignore_attachments") {
if err := updateAttachments(ctx, issue, ctx.FormStrings("files[]")); err != nil {
ctx.ServerError("UpdateAttachments", err)
return
}
}
rctx := renderhelper.NewRenderContextRepoComment(ctx, ctx.Repo.Repository)
content, err := markdown.RenderString(rctx, issue.Content)
if err != nil {
ctx.ServerError("RenderString", err)
return
}
ctx.JSON(http.StatusOK, map[string]any{
"content": content,
"contentVersion": issue.ContentVersion,
"attachments": attachmentsHTML(ctx, issue.Attachments, issue.Content),
2015-08-20 04:31:28 +08:00
})
}
// UpdateIssueDeadline updates an issue deadline
func UpdateIssueDeadline(ctx *context.Context) {
issue, err := issues_model.GetIssueByIndex(ctx, ctx.Repo.Repository.ID, ctx.PathParamInt64(":index"))
2016-07-26 02:48:17 +08:00
if err != nil {
if issues_model.IsErrIssueNotExist(err) {
ctx.NotFound("GetIssueByIndex", err)
} else {
ctx.Error(http.StatusInternalServerError, "GetIssueByIndex", err.Error())
}
2016-07-26 02:48:17 +08:00
return
}
if !ctx.Repo.CanWriteIssuesOrPulls(issue.IsPull) {
ctx.Error(http.StatusForbidden, "", "Not repo writer")
return
}
deadlineUnix, _ := common.ParseDeadlineDateToEndOfDay(ctx.FormString("deadline"))
if err := issues_model.UpdateIssueDeadline(ctx, issue, deadlineUnix, ctx.Doer); err != nil {
ctx.Error(http.StatusInternalServerError, "UpdateIssueDeadline", err.Error())
return
}
ctx.JSONRedirect("")
}
// UpdateIssueMilestone change issue's milestone
func UpdateIssueMilestone(ctx *context.Context) {
issues := getActionIssues(ctx)
if ctx.Written() {
2016-07-26 02:48:17 +08:00
return
}
milestoneID := ctx.FormInt64("id")
for _, issue := range issues {
oldMilestoneID := issue.MilestoneID
if oldMilestoneID == milestoneID {
continue
}
issue.MilestoneID = milestoneID
if err := issue_service.ChangeMilestoneAssign(ctx, issue, ctx.Doer, oldMilestoneID); err != nil {
ctx.ServerError("ChangeMilestoneAssign", err)
return
}
}
ctx.JSONOK()
}
// UpdateIssueAssignee change issue's or pull's assignee
func UpdateIssueAssignee(ctx *context.Context) {
issues := getActionIssues(ctx)
if ctx.Written() {
2016-07-26 02:48:17 +08:00
return
}
assigneeID := ctx.FormInt64("id")
action := ctx.FormString("action")
for _, issue := range issues {
switch action {
case "clear":
if err := issue_service.DeleteNotPassedAssignee(ctx, issue, ctx.Doer, []*user_model.User{}); err != nil {
ctx.ServerError("ClearAssignees", err)
return
}
default:
assignee, err := user_model.GetUserByID(ctx, assigneeID)
if err != nil {
ctx.ServerError("GetUserByID", err)
return
}
valid, err := access_model.CanBeAssigned(ctx, assignee, issue.Repo, issue.IsPull)
if err != nil {
ctx.ServerError("canBeAssigned", err)
return
}
if !valid {
ctx.ServerError("canBeAssigned", repo_model.ErrUserDoesNotHaveAccessToRepo{UserID: assigneeID, RepoName: issue.Repo.Name})
return
}
_, _, err = issue_service.ToggleAssigneeWithNotify(ctx, issue, ctx.Doer, assigneeID)
if err != nil {
ctx.ServerError("ToggleAssignee", err)
return
}
}
}
ctx.JSONOK()
2016-07-26 02:48:17 +08:00
}
// ChangeIssueReaction create a reaction for issue
Move macaron to chi (#14293) Use [chi](https://github.com/go-chi/chi) instead of the forked [macaron](https://gitea.com/macaron/macaron). Since macaron and chi have conflicts with session share, this big PR becomes a have-to thing. According my previous idea, we can replace macaron step by step but I'm wrong. :( Below is a list of big changes on this PR. - [x] Define `context.ResponseWriter` interface with an implementation `context.Response`. - [x] Use chi instead of macaron, and also a customize `Route` to wrap chi so that the router usage is similar as before. - [x] Create different routers for `web`, `api`, `internal` and `install` so that the codes will be more clear and no magic . - [x] Use https://github.com/unrolled/render instead of macaron's internal render - [x] Use https://github.com/NYTimes/gziphandler instead of https://gitea.com/macaron/gzip - [x] Use https://gitea.com/go-chi/session which is a modified version of https://gitea.com/macaron/session and removed `nodb` support since it will not be maintained. **BREAK** - [x] Use https://gitea.com/go-chi/captcha which is a modified version of https://gitea.com/macaron/captcha - [x] Use https://gitea.com/go-chi/cache which is a modified version of https://gitea.com/macaron/cache - [x] Use https://gitea.com/go-chi/binding which is a modified version of https://gitea.com/macaron/binding - [x] Use https://github.com/go-chi/cors instead of https://gitea.com/macaron/cors - [x] Dropped https://gitea.com/macaron/i18n and make a new one in `code.gitea.io/gitea/modules/translation` - [x] Move validation form structs from `code.gitea.io/gitea/modules/auth` to `code.gitea.io/gitea/modules/forms` to avoid dependency cycle. - [x] Removed macaron log service because it's not need any more. **BREAK** - [x] All form structs have to be get by `web.GetForm(ctx)` in the route function but not as a function parameter on routes definition. - [x] Move Git HTTP protocol implementation to use routers directly. - [x] Fix the problem that chi routes don't support trailing slash but macaron did. - [x] `/api/v1/swagger` now will be redirect to `/api/swagger` but not render directly so that `APIContext` will not create a html render. Notices: - Chi router don't support request with trailing slash - Integration test `TestUserHeatmap` maybe mysql version related. It's failed on my macOS(mysql 5.7.29 installed via brew) but succeed on CI. Co-authored-by: 6543 <6543@obermui.de>
2021-01-26 23:36:53 +08:00
func ChangeIssueReaction(ctx *context.Context) {
form := web.GetForm(ctx).(*forms.ReactionForm)
issue := GetActionIssue(ctx)
if ctx.Written() {
return
}
if !ctx.IsSigned || (ctx.Doer.ID != issue.PosterID && !ctx.Repo.CanReadIssuesOrPulls(issue.IsPull)) {
if log.IsTrace() {
if ctx.IsSigned {
issueType := "issues"
if issue.IsPull {
issueType = "pulls"
}
log.Trace("Permission Denied: User %-v not the Poster (ID: %d) and cannot read %s in Repo %-v.\n"+
"User in Repo has Permissions: %-+v",
ctx.Doer,
Rewrite logger system (#24726) ## ⚠️ Breaking The `log.<mode>.<logger>` style config has been dropped. If you used it, please check the new config manual & app.example.ini to make your instance output logs as expected. Although many legacy options still work, it's encouraged to upgrade to the new options. The SMTP logger is deleted because SMTP is not suitable to collect logs. If you have manually configured Gitea log options, please confirm the logger system works as expected after upgrading. ## Description Close #12082 and maybe more log-related issues, resolve some related FIXMEs in old code (which seems unfixable before) Just like rewriting queue #24505 : make code maintainable, clear legacy bugs, and add the ability to support more writers (eg: JSON, structured log) There is a new document (with examples): `logging-config.en-us.md` This PR is safer than the queue rewriting, because it's just for logging, it won't break other logic. ## The old problems The logging system is quite old and difficult to maintain: * Unclear concepts: Logger, NamedLogger, MultiChannelledLogger, SubLogger, EventLogger, WriterLogger etc * Some code is diffuclt to konw whether it is right: `log.DelNamedLogger("console")` vs `log.DelNamedLogger(log.DEFAULT)` vs `log.DelLogger("console")` * The old system heavily depends on ini config system, it's difficult to create new logger for different purpose, and it's very fragile. * The "color" trick is difficult to use and read, many colors are unnecessary, and in the future structured log could help * It's difficult to add other log formats, eg: JSON format * The log outputer doesn't have full control of its goroutine, it's difficult to make outputer have advanced behaviors * The logs could be lost in some cases: eg: no Fatal error when using CLI. * Config options are passed by JSON, which is quite fragile. * INI package makes the KEY in `[log]` section visible in `[log.sub1]` and `[log.sub1.subA]`, this behavior is quite fragile and would cause more unclear problems, and there is no strong requirement to support `log.<mode>.<logger>` syntax. ## The new design See `logger.go` for documents. ## Screenshot <details> ![image](https://github.com/go-gitea/gitea/assets/2114189/4462d713-ba39-41f5-bb08-de912e67e1ff) ![image](https://github.com/go-gitea/gitea/assets/2114189/b188035e-f691-428b-8b2d-ff7b2199b2f9) ![image](https://github.com/go-gitea/gitea/assets/2114189/132e9745-1c3b-4e00-9e0d-15eaea495dee) </details> ## TODO * [x] add some new tests * [x] fix some tests * [x] test some sub-commands (manually ....) --------- Co-authored-by: Jason Song <i@wolfogre.com> Co-authored-by: delvh <dev.lh@web.de> Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-22 06:35:11 +08:00
issue.PosterID,
issueType,
ctx.Repo.Repository,
ctx.Repo.Permission)
} else {
log.Trace("Permission Denied: Not logged in")
}
}
ctx.Error(http.StatusForbidden)
return
}
if ctx.HasError() {
ctx.ServerError("ChangeIssueReaction", errors.New(ctx.GetErrMsg()))
return
}
switch ctx.PathParam(":action") {
case "react":
reaction, err := issue_service.CreateIssueReaction(ctx, ctx.Doer, issue, form.Content)
if err != nil {
if issues_model.IsErrForbiddenIssueReaction(err) || errors.Is(err, user_model.ErrBlockedUser) {
ctx.ServerError("ChangeIssueReaction", err)
return
}
log.Info("CreateIssueReaction: %s", err)
break
}
// Reload new reactions
issue.Reactions = nil
if err = issue.LoadAttributes(ctx); err != nil {
log.Info("issue.LoadAttributes: %s", err)
break
}
log.Trace("Reaction for issue created: %d/%d/%d", ctx.Repo.Repository.ID, issue.ID, reaction.ID)
case "unreact":
if err := issues_model.DeleteIssueReaction(ctx, ctx.Doer.ID, issue.ID, form.Content); err != nil {
ctx.ServerError("DeleteIssueReaction", err)
return
}
// Reload new reactions
issue.Reactions = nil
if err := issue.LoadAttributes(ctx); err != nil {
log.Info("issue.LoadAttributes: %s", err)
break
}
log.Trace("Reaction for issue removed: %d/%d", ctx.Repo.Repository.ID, issue.ID)
default:
ctx.NotFound(fmt.Sprintf("Unknown action %s", ctx.PathParam(":action")), nil)
return
}
if len(issue.Reactions) == 0 {
ctx.JSON(http.StatusOK, map[string]any{
"empty": true,
"html": "",
})
return
}
html, err := ctx.RenderToHTML(tplReactions, map[string]any{
"ActionURL": fmt.Sprintf("%s/issues/%d/reactions", ctx.Repo.RepoLink, issue.Index),
"Reactions": issue.Reactions.GroupByType(),
})
if err != nil {
ctx.ServerError("ChangeIssueReaction.HTMLString", err)
return
}
ctx.JSON(http.StatusOK, map[string]any{
"html": html,
})
}
// GetIssueAttachments returns attachments for the issue
func GetIssueAttachments(ctx *context.Context) {
issue := GetActionIssue(ctx)
if ctx.Written() {
return
}
attachments := make([]*api.Attachment, len(issue.Attachments))
for i := 0; i < len(issue.Attachments); i++ {
attachments[i] = convert.ToAttachment(ctx.Repo.Repository, issue.Attachments[i])
}
ctx.JSON(http.StatusOK, attachments)
}
func updateAttachments(ctx *context.Context, item any, files []string) error {
var attachments []*repo_model.Attachment
switch content := item.(type) {
case *issues_model.Issue:
attachments = content.Attachments
case *issues_model.Comment:
attachments = content.Attachments
default:
return fmt.Errorf("unknown Type: %T", content)
}
for i := 0; i < len(attachments); i++ {
if util.SliceContainsString(files, attachments[i].UUID) {
continue
}
if err := repo_model.DeleteAttachment(ctx, attachments[i], true); err != nil {
return err
}
}
var err error
if len(files) > 0 {
switch content := item.(type) {
case *issues_model.Issue:
err = issues_model.UpdateIssueAttachments(ctx, content.ID, files)
case *issues_model.Comment:
err = content.UpdateAttachments(ctx, files)
default:
return fmt.Errorf("unknown Type: %T", content)
}
if err != nil {
return err
}
}
switch content := item.(type) {
case *issues_model.Issue:
content.Attachments, err = repo_model.GetAttachmentsByIssueID(ctx, content.ID)
case *issues_model.Comment:
content.Attachments, err = repo_model.GetAttachmentsByCommentID(ctx, content.ID)
default:
return fmt.Errorf("unknown Type: %T", content)
}
return err
}
func attachmentsHTML(ctx *context.Context, attachments []*repo_model.Attachment, content string) template.HTML {
attachHTML, err := ctx.RenderToHTML(tplAttachment, map[string]any{
"ctxData": ctx.Data,
"Attachments": attachments,
"Content": content,
})
if err != nil {
ctx.ServerError("attachmentsHTML.HTMLString", err)
return ""
}
return attachHTML
}
// handleMentionableAssigneesAndTeams gets all teams that current user can mention, and fills the assignee users to the context data
func handleMentionableAssigneesAndTeams(ctx *context.Context, assignees []*user_model.User) {
// TODO: need to figure out how many places this is really used, and rename it to "MentionableAssignees"
// at the moment it is used on the issue list page, for the markdown editor mention
ctx.Data["Assignees"] = assignees
if ctx.Doer == nil || !ctx.Repo.Owner.IsOrganization() {
return
}
var isAdmin bool
var err error
var teams []*organization.Team
org := organization.OrgFromUser(ctx.Repo.Owner)
// Admin has super access.
if ctx.Doer.IsAdmin {
isAdmin = true
} else {
isAdmin, err = org.IsOwnedBy(ctx, ctx.Doer.ID)
if err != nil {
ctx.ServerError("IsOwnedBy", err)
return
}
}
if isAdmin {
teams, err = org.LoadTeams(ctx)
if err != nil {
ctx.ServerError("LoadTeams", err)
return
}
} else {
teams, err = org.GetUserTeams(ctx, ctx.Doer.ID)
if err != nil {
ctx.ServerError("GetUserTeams", err)
return
}
}
ctx.Data["MentionableTeams"] = teams
ctx.Data["MentionableTeamsOrg"] = ctx.Repo.Owner.Name
Add context cache as a request level cache (#22294) To avoid duplicated load of the same data in an HTTP request, we can set a context cache to do that. i.e. Some pages may load a user from a database with the same id in different areas on the same page. But the code is hidden in two different deep logic. How should we share the user? As a result of this PR, now if both entry functions accept `context.Context` as the first parameter and we just need to refactor `GetUserByID` to reuse the user from the context cache. Then it will not be loaded twice on an HTTP request. But of course, sometimes we would like to reload an object from the database, that's why `RemoveContextData` is also exposed. The core context cache is here. It defines a new context ```go type cacheContext struct { ctx context.Context data map[any]map[any]any lock sync.RWMutex } var cacheContextKey = struct{}{} func WithCacheContext(ctx context.Context) context.Context { return context.WithValue(ctx, cacheContextKey, &cacheContext{ ctx: ctx, data: make(map[any]map[any]any), }) } ``` Then you can use the below 4 methods to read/write/del the data within the same context. ```go func GetContextData(ctx context.Context, tp, key any) any func SetContextData(ctx context.Context, tp, key, value any) func RemoveContextData(ctx context.Context, tp, key any) func GetWithContextCache[T any](ctx context.Context, cacheGroupKey string, cacheTargetID any, f func() (T, error)) (T, error) ``` Then let's take a look at how `system.GetString` implement it. ```go func GetSetting(ctx context.Context, key string) (string, error) { return cache.GetWithContextCache(ctx, contextCacheKey, key, func() (string, error) { return cache.GetString(genSettingCacheKey(key), func() (string, error) { res, err := GetSettingNoCache(ctx, key) if err != nil { return "", err } return res.SettingValue, nil }) }) } ``` First, it will check if context data include the setting object with the key. If not, it will query from the global cache which may be memory or a Redis cache. If not, it will get the object from the database. In the end, if the object gets from the global cache or database, it will be set into the context cache. An object stored in the context cache will only be destroyed after the context disappeared.
2023-02-15 21:37:34 +08:00
ctx.Data["MentionableTeamsOrgAvatar"] = ctx.Repo.Owner.AvatarLink(ctx)
}