1
0
mirror of https://github.com/go-gitea/gitea.git synced 2024-12-15 17:49:20 +08:00
Commit Graph

75 Commits

Author SHA1 Message Date
Lunny Xiao
98d9a71ffe
Trim title before insert/update to database to match the size requirements of database ()
Fix 
2024-11-14 07:19:14 +00:00
ChristopherHX
f888e45432
Harden runner updateTask and updateLog api ()
Per proposal https://github.com/go-gitea/gitea/issues/32461
2024-11-11 04:58:37 +00:00
Lunny Xiao
24b83ff63e
Fix milestone deadline and date related problems ()
Use zero instead of 9999-12-31 for deadline
Fix 

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
2024-11-05 07:46:40 +00:00
Zettat123
9116665e9c
Always update expiration time when creating an artifact ()
Fix 
2024-10-17 09:05:38 +00:00
FuXiaoHei
df27846628
Show latest run when visit /run/latest ()
Proposal from
https://github.com/go-gitea/gitea/issues/27911#issuecomment-2271982172

When visit latest run path, such as
`/{user}/{repo}/actions/runs/latest`. It renders latest run instead of
index=0 currently.
2024-08-10 08:40:41 +08:00
Jason Song
33cc5837a6
Support compression for Actions logs ()
Support compression for Actions logs to save storage space and
bandwidth. Inspired by
https://github.com/go-gitea/gitea/issues/24256#issuecomment-1521153015

The biggest challenge is that the compression format should support
[seekable](https://github.com/facebook/zstd/blob/dev/contrib/seekable_format/zstd_seekable_compression_format.md).
So when users are viewing a part of the log lines, Gitea doesn't need to
download the whole compressed file and decompress it.

That means gzip cannot help here. And I did research, there aren't too
many choices, like bgzip and xz, but I think zstd is the most popular
one. It has an implementation in Golang with
[zstd](https://github.com/klauspost/compress/tree/master/zstd) and
[zstd-seekable-format-go](https://github.com/SaveTheRbtz/zstd-seekable-format-go),
and what is better is that it has good compatibility: a seekable format
zstd file can be read by a regular zstd reader.

This PR introduces a new package `zstd` to combine and wrap the two
packages, to provide a unified and easy-to-use API.

And a new setting `LOG_COMPRESSION` is added to the config, although I
don't see any reason why not to use compression, I think's it's a good
idea to keep the default with `none` to be consistent with old versions.

`LOG_COMPRESSION` takes effect for only new log files, it adds `.zst` as
an extension to the file name, so Gitea can determine if it needs
decompression according to the file name when reading. Old files will
keep the format since it's not worth converting them, as they will be
cleared after .

<img width="541" alt="image"
src="https://github.com/user-attachments/assets/e9598764-a4e0-4b68-8c2b-f769265183c9">
2024-08-09 10:10:30 +08:00
Jason Song
687c118248
Clear up old Actions logs ()
Part of .

Clear up old action logs to free up storage space.

Users will see a message indicating that the log has been cleared if
they view old tasks.

<img width="1361" alt="image"
src="https://github.com/user-attachments/assets/9f0f3a3a-bc5a-402f-90ca-49282d196c22">

Docs: https://gitea.com/gitea/docs/pulls/40

---------

Co-authored-by: silverwind <me@silverwind.io>
2024-08-02 00:42:08 +00:00
Jason Song
21a73ae642
Use UTC as default timezone when schedule Actions cron tasks ()
Fix .

According to the
[doc](https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions#onschedule)
of GitHub Actions, The timezone for cron should be UTC, not the local
timezone. And Gitea Actions doesn't have any reasons to change this, so
I think it's a bug.

However, Gitea Actions has extended the syntax, as it supports
descriptors like `@weekly` and `@every 5m`, and supports specifying the
timezone like `TZ=UTC 0 10 * * *`. So we can make it use UTC only when
the timezone is not specified, to be compatible with GitHub Actions, and
also respect the user's specified.

It does break the feature because the times to run tasks would be
changed, and it may confuse users. So I don't think we should backport
this.

## ⚠️ BREAKING ⚠️

If the server's local time zone is not UTC, a scheduled task would run
at a different time after upgrading Gitea to this version.
2024-08-01 10:02:46 +00:00
Jason Song
a33e74d40d
Clarify Actions resources ownership ()
Fix .

Also related to .

Some Actions resources could has different types of ownership. It could
be:

- global: all repos and orgs/users can use it.
- org/user level: only the org/user can use it.
- repo level: only the repo can use it.

There are two ways to distinguish org/user level from repo level:
1. `{owner_id: 1, repo_id: 2}` for repo level, and `{owner_id: 1,
repo_id: 0}` for org level.
2. `{owner_id: 0, repo_id: 2}` for repo level, and `{owner_id: 1,
repo_id: 0}` for org level.

The first way seems more reasonable, but it may not be true. The point
is that although a resource, like a runner, belongs to a repo (it can be
used by the repo), the runner doesn't belong to the repo's org (other
repos in the same org cannot use the runner). So, the second method
makes more sense.

And the first way is not user-friendly to query, we must set the repo id
to zero to avoid wrong results.

So,  should be right. And the most simple way to fix  is
just:

```diff
-	shared.GetRegistrationToken(ctx, ctx.Repo.Repository.OwnerID, ctx.Repo.Repository.ID)
+	shared.GetRegistrationToken(ctx, 0, ctx.Repo.Repository.ID)
```

However, it is quite intuitive to set both owner id and repo id since
the repo belongs to the owner. So I prefer to be compatible with it. If
we get both owner id and repo id not zero when creating or finding, it's
very clear that the caller want one with repo level, but set owner id
accidentally. So it's OK to accept it but fix the owner id to zero.
2024-08-01 09:04:04 +00:00
Jason Song
572fa55fbc
Drop IDOrderDesc for listing Actions task and always order by id DESC ()
Close 

Just follow what `FindRunOptions` and `FindScheduleOptions` do.
2024-05-31 02:30:02 +00:00
silverwind
9b2536b78f
Update misspell to 0.5.1 and add misspellings.csv ()
Misspell 0.5.0 supports passing a csv file to extend the list of
misspellings, so I added some common ones from the codebase. There is at
least one typo in a API response so we need to decided whether to revert
that and then likely remove the dict entry.
2024-04-27 08:03:49 +00:00
yp05327
2a3906d755
Improve job commit description ()
Fix https://github.com/go-gitea/gitea/issues/30567

When job is a schedule:

![image](https://github.com/go-gitea/gitea/assets/18380374/b07e9d43-e8b7-4ee2-87b3-a7050c3a8ca5)
When it is a normal one:

![image](https://github.com/go-gitea/gitea/assets/18380374/0d58dab9-74bb-421b-8952-0578cdf21a52)

also add a 'space' behind  `:`

![image](https://github.com/go-gitea/gitea/assets/18380374/4cebece0-bfe6-4ad9-b806-e5c49bb9be43)


![image](https://github.com/go-gitea/gitea/assets/18380374/02da7681-474b-4c0f-9dad-b6558f6cb484)

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-04-26 02:22:45 +00:00
sillyguodong
2f6b1c46a1
Interpolate runs-on with variables when scheduling tasks ()
Follow 
1. Interpolate runs-on with variables when scheduling tasks.
2. The `GetVariablesOfRun` function will check if the `Repo` of the run
is nil.

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2024-04-23 20:55:25 +02:00
Zettat123
30dd4beeee
Add a db consistency check to remove runners that do not belong to a repository ()
Follow 
2024-04-23 03:51:52 +00:00
yp05327
96d31fe0a8
Avoid user does not exist error when detecting schedule actions when the commit author is an external user ()
![image](https://github.com/go-gitea/gitea/assets/18380374/ddf6ee84-2242-49b9-b066-bd8429ba4d76)

When repo is a mirror, and commit author is an external user, then
`GetUserByEmail` will return error.

reproduce/test:
- mirror Gitea to your instance
- disable action and enable it again, this will trigger
`DetectAndHandleSchedules`

ps: also follow , it only fixed normal runs, not scheduled runs.
2024-04-11 15:11:32 +08:00
Lunny Xiao
b09687f1d1
Refactor more filterslice () 2024-04-10 04:18:41 +00:00
Jason Song
310e2517e5
Fix ambiguous id when fetch Actions tasks ()
Fix regression of .

```txt
time="2024-04-10T02:23:49Z" level=error msg="failed to fetch task" func="[fetchTask]" file="[poller.go:91]" error="unknown: rpc error: code = Internal desc = pick task: CreateTaskForRunner: Error 1052 (23000): Column 'id' in field list is ambiguous"
```
2024-04-10 02:57:43 +00:00
oliverpool
d547b53cca
Add container.FilterSlice function ()
Many places have the following logic:
```go
func (jobs ActionJobList) GetRunIDs() []int64 {
	ids := make(container.Set[int64], len(jobs))
	for _, j := range jobs {
		if j.RunID == 0 {
			continue
		}
		ids.Add(j.RunID)
	}
	return ids.Values()
}
```

this introduces a `container.FilterMapUnique` function, which reduces
the code above to:
```go
func (jobs ActionJobList) GetRunIDs() []int64 {
	return container.FilterMapUnique(jobs, func(j *ActionRunJob) (int64, bool) {
		return j.RunID, j.RunID != 0
	})
}
```
2024-04-09 20:27:30 +08:00
yp05327
d872ce006c
Avoid running action when action unit is disabled after workflows detected ()
Fix 

We only checking unit disabled when detecting workflows, but not in
runner `FetchTask`.
So if a workflow was detected when action unit is enabled, but disabled
later, `FetchTask` will still return these detected actions.

Global setting: repo.ENABLED and repository.`DISABLED_REPO_UNITS` will
not effect this.
2024-04-08 22:08:26 +08:00
sillyguodong
62b073e6f3
Add API for Variables ()
close 

---------

Co-authored-by: silverwind <me@silverwind.io>
2024-03-28 20:40:35 +00:00
Jason Song
b150ff0bab
Cancel previous runs of the same PR automatically ()
Follow . Also cancel previous runs for `pull_request_sync`.

It's not a bug since it original PR said "if the event is push".

The main change is
https://github.com/go-gitea/gitea/pull/29961/files#diff-08adda3f8ae0360937f46abb1f4418603bd3518522baa356be11c6c7ac4abcc3.

And also rename `CancelRunningJobs` to `CancelPreviousJobs` to make it
more clear.
2024-03-21 15:01:35 +08:00
sillyguodong
a1f5dd7677
Make runs-on support variable expression ()
As title.
Close issue: https://gitea.com/gitea/act_runner/issues/445
Follow: https://gitea.com/gitea/act/pulls/91

Move `getSecretsOfTask` and `getVariablesOfTask` under `models` because
of circular dependency issues.
2024-03-08 06:14:35 +00:00
6543
a3f05d0d98
remove util.OptionalBool and related functions ()
and migrate affected code

_last refactoring bits to replace **util.OptionalBool** with
**optional.Option[bool]**_
2024-03-02 16:42:31 +01:00
Nanguan Lin
db545b208b
Implement actions badge svgs ()
replace  
close 
The badge has two parts: label(workflow name) and message(action
status). 5 colors are provided with 7 statuses.
Color mapping:
```go
var statusColorMap = map[actions_model.Status]string{
	actions_model.StatusSuccess:   "#4c1",    // Green
	actions_model.StatusSkipped:   "#dfb317", // Yellow
	actions_model.StatusUnknown:   "#97ca00", // Light Green
	actions_model.StatusFailure:   "#e05d44", // Red
	actions_model.StatusCancelled: "#fe7d37", // Orange
	actions_model.StatusWaiting:   "#dfb317", // Yellow
	actions_model.StatusRunning:   "#dfb317", // Yellow
	actions_model.StatusBlocked:   "#dfb317", // Yellow
}
```
preview:

![1](https://github.com/go-gitea/gitea/assets/70063547/5465cbaf-23cd-4437-9848-2738c3cb8985)

![2](https://github.com/go-gitea/gitea/assets/70063547/ec393d26-c6e6-4d38-b72c-51f2494c5e71)

![3](https://github.com/go-gitea/gitea/assets/70063547/3edb4fdf-1b08-4a02-ab2a-6bdd7f532fb2)

![4](https://github.com/go-gitea/gitea/assets/70063547/8c189de2-2169-4251-b115-0e39a52f3df8)

![5](https://github.com/go-gitea/gitea/assets/70063547/3fe22c73-c2d7-4fec-9ea4-c501a1e4e3bd)

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
Co-authored-by: delvh <dev.lh@web.de>
2024-02-27 18:56:18 +01:00
FuXiaoHei
67adc5c1dc
Artifact deletion in actions ui ()
Add deletion link in runs view page.
Fix 


![image](https://github.com/go-gitea/gitea/assets/2142787/aa65a4ab-f434-4deb-b953-21e63c212033)

When click deletion button. It marks this artifact `need-delete`.

This artifact would be deleted when actions cleanup cron task.
2024-02-18 10:33:50 +00:00
wxiaoguang
f3eb835886
Refactor locale&string&template related code ()
Clarify when "string" should be used (and be escaped), and when
"template.HTML" should be used (no need to escape)

And help PRs like   , to render the error messages correctly.
2024-02-14 21:48:45 +00:00
yp05327
07ba4d9f87
Fix incorrect action duration time when rerun the job before executed once ()
Fix 
Reason was mentioned here:
https://github.com/go-gitea/gitea/issues/28323#issuecomment-1841867298

### Changes: (maybe breaking)
We can rerun jobs in Gitea, so there will be some problems in
calculating duration time.
In this PR, I use the exist `Started` and `Stopped` column to record the
last run time instead of the total time,
and add a new `PreviousDuration` column to record the previous duration
time.
You can also check the cost time of last run:

![image](https://github.com/go-gitea/gitea/assets/18380374/2ca39145-2c92-401a-b78b-43164f7ae061)
2024-01-19 14:05:49 +00:00
Lunny Xiao
97292da960
Fix schedule tasks bugs ()
Fix  

This PR fix the possible bugs about actions schedule.

## The Changes

- Move `UpdateRepositoryUnit` and `SetRepoDefaultBranch` from models to
service layer
- Remove schedules plan from database and cancel waiting & running
schedules tasks in this repository when actions unit has been disabled
or global disabled.
- Remove schedules plan from database and cancel waiting & running
schedules tasks in this repository when default branch changed.
2024-01-12 21:50:38 +00:00
delvh
778ad795fd
Refactor deletion ()
Introduce the new generic deletion methods
- `func DeleteByID[T any](ctx context.Context, id int64) (int64, error)`
- `func DeleteByIDs[T any](ctx context.Context, ids ...int64) error`
- `func Delete[T any](ctx context.Context, opts FindOptions) (int64,
error)`

So, we no longer need any specific deletion method and can just use
the generic ones instead.

Replacement of 

Closes 

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-12-25 21:25:29 +01:00
Jean-Baptiste Gomond
d0f24ff4ca
Added instance-level variables ()
This PR adds instance-level variables, and so closes 



![gitea_instance_variables_1](https://github.com/go-gitea/gitea/assets/8344487/ad409cd4-ce36-4c84-a764-34451b0fb63a)

![gitea_instance_variables_2](https://github.com/go-gitea/gitea/assets/8344487/426f0965-dec6-4560-948c-067cdeddd720)

![gitea_instance_variables_3](https://github.com/go-gitea/gitea/assets/8344487/cf1d7776-4938-4825-922e-cbbbf28a5f33)
2023-12-25 07:28:59 +00:00
yp05327
4ea522fecf
Only check online runner when detecting matching runners in workflows ()
Mentioned:
[](https://github.com/go-gitea/gitea/issues/28277#issuecomment-1831325276)

We should only check online runner when detecting matching runners in
workflows,
as if runner is not online, the workflow will not run.


![image](https://github.com/go-gitea/gitea/assets/18380374/11855e9d-7241-4b7a-b8d7-49dbb94ba1c5)
2023-12-18 16:06:19 +00:00
yp05327
f4561c44b1
Fix incorrect run order of action jobs ()
When we pick up a job, all waiting jobs should firstly be ordered by
update time,
otherwise when there's a running job, if I rerun an older job, the older
job will run first, as it's id is smaller.
2023-12-06 22:10:05 +00:00
Lunny Xiao
df1e7d0067
Use db.Find instead of writing methods for every object ()
For those simple objects, it's unnecessary to write the find and count
methods again and again.
2023-11-24 03:49:41 +00:00
yp05327
976a9926d7
Fix no ActionTaskOutput table waring ()
Reproduce:
- Create a new Gitea instance
- Register a runner
- Create a repo and add a workflow
- Check the log, you will see warnings:

![image](https://github.com/go-gitea/gitea/assets/18380374/5f1278e0-114b-48bc-8113-8ba1404d9975)
It comes from:

![image](https://github.com/go-gitea/gitea/assets/18380374/c2807831-e137-4229-9536-87f6114c8a5b)

The reason is that we forgot registering `ActionTaskOutput` model.
So `action_table_output` table will be missing in your db.
2023-11-21 08:02:51 +00:00
Earl Warren
59f9ef9fee
Remove action runners on user deletion ()
- On user deletion, delete action runners that the user has created.
- Add a database consistency check to remove action runners that have
nonexistent belonging owner.
- Resolves https://codeberg.org/forgejo/forgejo/issues/1720

(cherry picked from commit 009ca7223dab054f7f760b7ccae69e745eebfabb)

Co-authored-by: Gusted <postmaster@gusted.xyz>
2023-11-05 13:48:32 +01:00
silverwind
ce83609ff6
Upgrade to golangci-lint@v1.55.0 ()
https://github.com/golangci/golangci-lint/releases/tag/v1.55.0
2023-10-24 02:54:59 +00:00
Nanguan Lin
80a0ab350b
Add unit tests for action runner token ()
In case the behavior of the register token changes.
2023-10-19 07:24:24 +00:00
Jason Song
0a2b93d411
Fix typo "GetLatestRunnerToken" () 2023-10-18 15:52:44 +00:00
JakobDev
ebe803e514
Penultimate round of db.DefaultContext refactor ()
Part of 

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-10-11 04:24:07 +00:00
Lunny Xiao
3b2da7e4ae
Redefine the meaning of column is_active to make Actions Registration Token generation easier ()
Partially Fix 

This PR redefined the meaning of column `is_active` in table
`action_runner_token`.
Before this PR, `is_active` means whether it has been used by any
runner. If it's true, other runner cannot use it to register again.

In this PR, `is_active` means whether it's validated to be used to
register runner. And if it's true, then it can be used to register
runners until it become false. When creating a new `is_active` register
token, any previous tokens will be set `is_active` to false.
2023-09-27 05:37:48 +00:00
JakobDev
f91dbbba98
Next round of db.DefaultContext refactor ()
Part of 
2023-09-16 14:39:12 +00:00
jladbrook
e33f112e01
resolve issue with sort icons on admin/users and admin/runners ()
Fixes  to avoid the sort icon changing the table header over
multiple lines and adds missing sort icons on the runners page.
2023-09-12 12:26:51 +00:00
Lunny Xiao
9c0a3532a4
Add a new column schedule_id for action_run to track ()
Fix  

And the UI now will display it's scheduled but not triggered by a push.

<img width="954" alt="图片"
src="https://github.com/go-gitea/gitea/assets/81045/d211845c-457e-4c3e-af1f-a0d654d3f365">
2023-09-08 23:01:19 +08:00
CaiCandong
a78c2eae24
Replace util.SliceXxx with slices.Xxx () 2023-09-07 09:37:47 +00:00
FuXiaoHei
460a2b0edf
Artifacts retention and auto clean up ()
Currently, Artifact does not have an expiration and automatic cleanup
mechanism, and this feature needs to be added. It contains the following
key points:

- [x] add global artifact retention days option in config file. Default
value is 90 days.
- [x] add cron task to clean up expired artifacts. It should run once a
day.
- [x] support custom retention period from `retention-days: 5` in
`upload-artifact@v3`.
- [x] artifacts link in actions view should be non-clickable text when
expired.
2023-09-06 07:41:06 +00:00
Lunny Xiao
0d55f64e6c
chore(actions): support cron schedule task ()
Replace  

1. only support the default branch in the repository setting.
2. autoload schedule data from the schedule table after starting the
service.
3. support specific syntax like `@yearly`, `@monthly`, `@weekly`,
`@daily`, `@hourly`

## How to use

See the [GitHub Actions
document](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#schedule)
for getting more detailed information.

```yaml
on:
  schedule:
    - cron: '30 5 * * 1,3'
    - cron: '30 5 * * 2,4'

jobs:
  test_schedule:
    runs-on: ubuntu-latest
    steps:
      - name: Not on Monday or Wednesday
        if: github.event.schedule != '30 5 * * 1,3'
        run: echo "This step will be skipped on Monday and Wednesday"
      - name: Every time
        run: echo "This step will always run"
```

Signed-off-by: Bo-Yi.Wu <appleboy.tw@gmail.com>

---------


Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-08-24 03:06:51 +00:00
Jason Song
8cf3b61fb9
Add optimistic lock to ActionRun table ()
Should fix .

How xorm works: https://xorm.io/docs/chapter-06/1.lock/

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2023-08-21 14:07:52 +00:00
cassiozareck
952882fe4a
Fix log typo in task.go ()
Signed-off-by: cassiozareck <cassiomilczareck@gmail.com>
2023-08-05 11:43:03 +02:00
yp05327
d817b199ca
Fix wrong workflow status when rerun a job in an already finished workflow ()
Before:

![image](https://github.com/go-gitea/gitea/assets/18380374/fb687592-b117-4cd5-b076-2ca5ca847ea4)
After:

![image](https://github.com/go-gitea/gitea/assets/18380374/c9b0683e-e81d-410b-8c35-fbe54327fab4)

After workflow finished, if you rerun a single job, the workflow status
will become to `Running` which is not correct as no jobs are running in
this workflow.
2023-07-25 09:13:36 +00:00
Bo-Yi Wu
44781f9f5c
Implement auto-cancellation of concurrent jobs if the event is push ()
- cancel running jobs if the event is push
- Add a new function `CancelRunningJobs` to cancel all running jobs of a
run
- Update `FindRunOptions` struct to include `Ref` field and update its
condition in `toConds` function
- Implement auto cancellation of running jobs in the same workflow in
`notify` function

related task: https://github.com/go-gitea/gitea/pull/22751/

---------

Signed-off-by: Bo-Yi Wu <appleboy.tw@gmail.com>
Signed-off-by: appleboy <appleboy.tw@gmail.com>
Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: delvh <dev.lh@web.de>
2023-07-25 11:15:55 +08:00