Compare commits

...

95 Commits

Author SHA1 Message Date
Zettat123
f913d90ab6 Update CHANGELOG for v1.25.5 (#36885)
Wait

- ~~#36888~~
2026-03-12 19:26:33 -06:00
Zettat123
156d9ffb23 Update toolchain to 1.25.8 for v1.25 (#36888)
> go1.25.8 (released 2026-03-05) includes security fixes to the
html/template, net/url, and os packages, as well as bug fixes to the go
command, the compiler, and the os package. See the [Go 1.25.8
milestone](https://github.com/golang/go/issues?q=milestone%3AGo1.25.8+label%3ACherryPickApproved)
on our issue tracker for details.
2026-03-11 17:37:33 -07:00
Lunny Xiao
96515c0f20 Fix org permission API visibility checks for hidden members and private orgs (#36798) (#36841)
backport #36798 

- fix wrong parameter of HasOrgOrUserVisible in
routers/api/v1/org/org.go
- add integration tests covering the bug fix
- merge permissions API tests

---
Generated by a coding agent with Codex 5.2
2026-03-08 16:26:08 +00:00
Lunny Xiao
4f562da975 Fix non-admins unable to automerge PRs from forks (#36833) (#36843)
backport #36833 

Make `handlePullRequestAutoMerge` correctly check the permissions of the
merging user against pr.BaseRepo.

Co-authored-by: Michael Hoang <10492681+Enzime@users.noreply.github.com>
Co-authored-by: Michael Hoang <enzime@users.noreply.github.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2026-03-08 10:18:35 +00:00
Giteabot
76c539cd57 Fix bug to check whether user can update pull request branch or rebase branch (#36465) (#36838)
Backport #36465 by @lunny

When checking whether a user can update a pull request branch or perform
an update via rebase, a maintainer should inherit the pull request
author’s permissions if Allow maintainer edits is enabled.

Signed-off-by: Lunny Xiao <xiaolunwen@gmail.com>
Signed-off-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2026-03-08 03:23:54 +00:00
Lunny Xiao
f7e3569fab Add a git grep search timeout (#36809) (#36835)
Backport #36809
2026-03-07 20:40:16 +00:00
Copilot
b3290b62fc Backport: Make security-check informational only (#36681) (#36852)
Backport #36681

`security-check` (govulncheck) was failing CI on all PRs whenever
vulnerabilities existed in dependencies. Since
https://github.com/go-gitea/gitea/security/dependabot already surfaces
this information, the check should be informational only.

- **`Makefile`**: Append `|| true` to the `security-check` target so
govulncheck output is preserved but non-zero exits no longer break CI.

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: silverwind <115237+silverwind@users.noreply.github.com>
2026-03-06 22:53:59 +00:00
Giteabot
f7ac507671 Fix dump release asset bug (#36799) (#36839)
Backport #36799 by @lunny

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: silverwind <me@silverwind.io>
2026-03-06 19:50:17 +00:00
Giteabot
e2517e0fa9 Fix forwarded proto handling for public URL detection (#36810) (#36836)
Backport #36810 by @lunny

- normalize `X-Forwarded-Proto`/related headers to accept only
`http`/`https`
- ignore malformed or injected scheme values to prevent spoofed
canonical URLs
- add tests covering malicious and multi-valued forwarded proto headers

---
Generated by a coding agent with Codex 5.2

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: silverwind <me@silverwind.io>
2026-03-06 19:02:50 +00:00
Giteabot
413074b1e1 Fix OAuth2 authorization code expiry and reuse handling (#36797) (#36851)
Backport #36797 by @lunny

- set OAuth2 authorization code `ValidUntil` on creation and add expiry
checks during exchange
- return a specific error when codes are invalidated twice to prevent
concurrent reuse
- add unit tests covering validity timestamps, expiration, and double
invalidation

---
Generate by a coding agent with Codex 5.2

Signed-off-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-06 10:33:20 -08:00
Lunny Xiao
3c46a3deb3 Fix bug when pushing mirror with wiki (#36795) (#36807)
Fix #36736
Backport #36795

Co-authored-by: ChristopherHX <christopher.homberger@web.de>
2026-03-06 16:26:34 +01:00
Giteabot
5552eff6e7 Fix artifacts v4 backend upload problems (#36805) (#36834)
Backport #36805 by @ChristopherHX

* Use base64.RawURLEncoding to avoid equal sign
  * using the nodejs package they seem to get lost
* Support uploads with unspecified length
* Support uploads with a single named blockid
  * without requiring a blockmap

Signed-off-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: ChristopherHX <christopher.homberger@web.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2026-03-06 14:22:53 +01:00
Lunny Xiao
f44f7bf2d3 upgrade to github.com/cloudflare/circl 1.6.3, svgo 4.0.1, markdownlint-cli 0.48.0 (#36840)
Backport #36837

---------

Co-authored-by: Christopher Homberger <christopher.homberger@web.de>
2026-03-06 12:55:33 +01:00
Giteabot
0f55eff0e7 Fix CRAN package version validation to allow more than 4 version components (#36813) (#36821)
Backport #36813

Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: wxiaoguang <2114189+wxiaoguang@users.noreply.github.com>
2026-03-04 09:29:22 +08:00
Giteabot
b3bc79262d Add validation constraints for repository creation fields (#36671) (#36757)
Backport #36671 by @lunny

Adds validation constraints to repository creation inputs, enforcing
max-length limits for labels/license/readme and enum validation for
trust model and object format. Updates both the API option struct and
the web form struct to keep validation consistent.

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2026-02-25 20:43:00 +00:00
Lunny Xiao
d1bd84f8cf Fix force push time-line commit comments of pull request (#36653) (#36717)
Backport #36653 

Fix #36647
Fix #25827
Fix #25870

Signed-off-by: silverwind <me@silverwind.io>
Signed-off-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-02-25 11:54:30 -08:00
Giteabot
19e36e8a70 Fix SVG height calculation in diff viewer (#36748) (#36750)
Backport #36748 by POPSuL

Fixes #36742

Co-authored-by: Viktor Suprun <popsul1993@gmail.com>
2026-02-26 00:46:35 +08:00
Giteabot
00566cc953 Fix track time list permission check (#36662) (#36744)
Backport #36662 by @lunny

Signed-off-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2026-02-25 07:57:47 -08:00
Giteabot
579615936c Fix path resolving (#36734) (#36746)
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2026-02-25 12:11:53 +08:00
Giteabot
2aee44cdd9 Prevent redirect bypasses via backslash-encoded paths (#36660) (#36716)
Backport #36660 by @lunny

This change tightens relative URL validation to reject raw backslashes
and `%5c` (encoded backslash), since browsers and URL normalizers can
treat backslashes as path separators. That normalization can turn
seemingly relative paths into scheme-relative URLs, creating
open-redirect risk.

Visiting below URL to reproduce the problem.

http://localhost:3000/user/login?redirect_to=/a/../\example.com

http://localhost:3000/user/login?redirect_to=/a/../%5cexample.com

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2026-02-23 01:59:59 +01:00
Giteabot
e7fca90a78 Fix get release draft permission check (#36659) (#36715)
Backport #36659 by @lunny

Draft release and it's attachments need a write permission to access.

Signed-off-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-02-22 22:09:07 +00:00
Lunny Xiao
3422318545 Fix push time bug (#36693) (#36713)
When display or search branch's pushed time, we should use
`updated_unix` rather than `commit_time`.

Fix #36633
Backport #36693

Signed-off-by: silverwind <me@silverwind.io>
Co-authored-by: silverwind <me@silverwind.io>
2026-02-22 22:27:40 +01:00
Giteabot
996cc12bf7 Add migration http transport for push/sync mirror lfs (#36665) (#36691)
Backport #36665 by @lunny

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2026-02-22 08:56:14 +00:00
Giteabot
99bc281856 Add some validation on values provided to USER_DISABLED_FEATURES and EXTERNAL_USER_DISABLED_FEATURES (#36688) (#36692) 2026-02-21 11:13:15 -05:00
Giteabot
8051056075 Fix track time issue id (#36664) (#36689)
Backport #36664 by @lunny

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2026-02-21 00:26:56 +00:00
Lunny Xiao
0b2f7575e7 Upgrade gogit to 5.16.5 (#36687)
Backport #36680
2026-02-20 15:02:38 -08:00
Giteabot
216cf96cd4 Fix bug the protected branch rule name is conflicted with renamed branch name (#36650) (#36661)
Backport #36650 by @lunny

Fix #36464

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2026-02-17 21:57:43 +00:00
Lunny Xiao
e927a86586 Fix a bug user could change another user's primary email (#36586) (#36607)
backport #36586
2026-02-14 14:06:59 +02:00
Giteabot
76b7306daa Fix bug when do LFS GC (#36500) (#36608)
Backport #36500 by @lunny

Fix #36448

Removed unnecessary parameters from the LFS GC process and switched to
an ORDER BY id ASC strategy with a last-ID cursor to avoid missing or
duplicating meta object IDs.

Signed-off-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-02-13 04:30:42 +00:00
Tyrone Yeh
8e412ababf Fix focus lost bugs in the Monaco editor (#36609)
…t focus (#36585)

Currently, pressing the space key in the Monaco editor scrolls the page
instead of inserting a space
if the editor is focused. This PR stops the space key event from
propagating to parent elements,
which prevents unwanted page scrolling while still allowing Monaco to
handle space input normally.

Changes:
 - disable Monaco editContext

No changes to default editor behavior are needed; Monaco automatically
inserts the space character.

Signed-off-by: silverwind <me@silverwind.io>
Co-authored-by: silverwind <me@silverwind.io>
2026-02-13 05:00:17 +01:00
Tyrone Yeh
4f1408cdcf fix(diff): reprocess htmx content after loading more files (#36568) (#36577) 2026-02-10 13:10:32 +08:00
Giteabot
5973437abb Add wrap to runner label list (#36565) (#36574)
Backport #36565 by @silverwind

Before: Label list forces runner table to become scrollable if there is
a large number of labels:

<img width="820" height="115" alt="Screenshot 2026-02-09 at 09 21 32"
src="https://github.com/user-attachments/assets/919a3b12-c8f6-48c4-bd42-d7e267faf107"
/>

After: Wrapped:

<img width="821" height="128" alt="Screenshot 2026-02-09 at 09 20 31"
src="https://github.com/user-attachments/assets/9f6d490c-1035-44be-97a7-20a1632dbe5e"
/>

Co-authored-by: silverwind <me@silverwind.io>
2026-02-10 04:47:12 +00:00
Giteabot
90843398ed fix: add dnf5 command for Fedora in RPM package instructions (#36527) (#36572)
Backport #36527 by @yshyuk

## Summary
Add support for Fedora 41+ which uses dnf5 with different command syntax
for adding repositories.

## Changes
- Added new locale key `packages.rpm.distros.fedora` for Fedora (dnf5)
- Added dnf5 command in RPM package template: `dnf config-manager
addrepo --from-repofile=<URL>`
- Kept existing dnf4 command (`--add-repo`) for RHEL/Rocky Linux
compatibility

## Background
Fedora 41+ uses dnf5 which has different syntax:
- **dnf4 (RHEL/Rocky):** `dnf config-manager --add-repo <URL>`
- **dnf5 (Fedora 41+):** `dnf config-manager addrepo
--from-repofile=<URL>`

Closes #35330

Co-authored-by: yshyuk <43194469+yshyuk@users.noreply.github.com>
2026-02-10 02:16:39 +01:00
Giteabot
9b3a9527ec Fix assignee sidebar links and empty placeholder (#36559) (#36563)
Backport #36559 by tyroneyeh

Co-authored-by: Tyrone Yeh <siryeh@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2026-02-09 03:31:38 +00:00
Giteabot
7477f85e47 Fix issues filter dropdown showing empty label scope section (#36535) (#36544)
Backport #36535 by tyroneyeh
2026-02-08 15:59:16 +00:00
wxiaoguang
4098032aa8 Fix various mermaid bugs (#36547) (#36552)
Backport #36547
2026-02-08 19:24:35 +08:00
Giteabot
dcce96c08d [SECURITY] fix: Adjust the toolchain version (#36537) (#36542)
Backport #36537 by @ZPascal

# Summary:

- Adjust the toolchain version to fix the security issues


```log
Vulnerability #1: GO-2026-4337
    Unexpected session resumption in crypto/tls
  More info: https://pkg.go.dev/vuln/GO-2026-4337
  Standard library
    Found in: crypto/tls@go1.25.6
    Fixed in: crypto/tls@go1.25.7
    Example traces found:
```

Signed-off-by: Pascal Zimmermann <pascal.zimmermann@theiotstudio.com>
Co-authored-by: Pascal Zimmermann <pascal.zimmermann@theiotstudio.com>
2026-02-06 23:00:52 +08:00
Giteabot
885f2b89d6 fix(packages/container): data race when uploading container blobs concurrently (#36524) (#36526)
Backport #36524 by @noeljackson

Fix data race when uploading container blobs concurrently

Co-authored-by: Noel Jackson <n@noeljackson.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2026-02-04 09:32:26 -08:00
Giteabot
57ce10c0ca Allow scroll propagation outside code editor (#36502) (#36510)
Backport #36502 by @lunny

Fix #28479

When scrolling inside the editor and the editor has already reached the
end of its scroll area, the browser does not continue scrolling. This is
inconvenient because users must move the cursor out of the editor to
scroll the page further.

This PR enables automatic switching between the editor’s scroll and the
browser’s scroll, allowing seamless continuous scrolling.

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2026-02-01 09:33:23 -08:00
Sebastian Ertz
25785041e7 Correct spacing between username and bot label (#36473) (#36484)
Backport #36473

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2026-01-30 05:47:46 +00:00
Giteabot
ff3d11034d [SECURITY] Toolchain Update to Go 1.25.6 (#36480) (#36487)
Backport #36480 by @ZPascal

## Overview
This PR updates the Go toolchain version from `1.25.5` to `1.25.6` for
the Gitea project.

## Changes

### Toolchain Update
- **Go Toolchain**: Updated from `go1.25.5` to `go1.25.6`

This is a minor toolchain version bump that ensures the project uses the
latest patch release of Go 1.25.

## Security Improvements

While this PR primarily addresses the toolchain update, the project
maintains a strong security posture through:

### Current Security Measures
```log
Vulnerability #1: GO-2026-4342                                                                                                                                                                                                      
    Excessive CPU consumption when building archive index in archive/zip
  More info: https://pkg.go.dev/vuln/GO-2026-4342
  Standard library
    Found in: archive/zip@go1.25.5
    Fixed in: archive/zip@go1.25.6
    Example traces found:
      #1: modules/packages/nuget/metadata.go:217:25: nuget.ParseNuspecMetaData calls zip.Reader.Open                                                                                                                                

Vulnerability #2: GO-2026-4341
    Memory exhaustion in query parameter parsing in net/url
  More info: https://pkg.go.dev/vuln/GO-2026-4341
  Standard library
    Found in: net/url@go1.25.5
    Fixed in: net/url@go1.25.6
    Example traces found:
      #1: modules/storage/minio.go:284:34: storage.MinioStorage.URL calls url.ParseQuery                                                                                                                                            
      #2: routers/api/v1/repo/action.go:1640:29: repo.DownloadArtifactRaw calls url.URL.Query

Vulnerability #3: GO-2026-4340
    Handshake messages may be processed at the incorrect encryption level in
    crypto/tls
  More info: https://pkg.go.dev/vuln/GO-2026-4340
  Standard library
    Found in: crypto/tls@go1.25.5
    Fixed in: crypto/tls@go1.25.6
    Example traces found:
      #1: services/auth/source/ldap/source_search.go:129:25: ldap.dial calls ldap.Conn.StartTLS, which calls tls.Conn.Handshake                                                                                                     
      #2: modules/graceful/server.go:156:14: graceful.Server.Serve calls http.Server.Serve, which eventually calls tls.Conn.HandshakeContext
      #3: modules/lfs/content_store.go:132:27: lfs.hashingReader.Read calls tls.Conn.Read
      #4: modules/proxyprotocol/conn.go:91:21: proxyprotocol.Conn.Write calls tls.Conn.Write
      #5: modules/session/virtual.go:168:39: session.VirtualStore.Release calls couchbase.CouchbaseProvider.Exist, which eventually calls tls.Dial
      #6: services/auth/source/ldap/source_search.go:120:22: ldap.dial calls ldap.DialTLS, which calls tls.DialWithDialer
      #7: services/migrations/gogs.go:114:34: migrations.client calls http.Transport.RoundTrip, which eventually calls tls.Dialer.DialContext
```

Co-authored-by: Pascal Zimmermann <pascal.zimmermann@theiotstudio.com>
2026-01-29 21:18:21 -08:00
Giteabot
750649c1ef Fix oauth2 s256 (#36462) (#36477)
Backport #36462 by @lunny

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2026-01-28 12:37:39 -08:00
Lunny Xiao
eb95bbc1fd Add missing changelog for v1.25.4 (#36433) 2026-01-23 06:35:34 +01:00
Lunny Xiao
369830bada Release notes for 1.25.4 (#36385)
---------

Signed-off-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Thomas Beutlich <115483027+thbeu@users.noreply.github.com>
2026-01-22 00:57:25 +00:00
Tyrone Yeh
d7d6533311 Fix markdown newline handling during IME composition (gitea#36421) (#36424)
Backport of #36421
2026-01-21 16:56:39 -08:00
Giteabot
c326369f47 Allow foreachref parse max tokens from 4*64KB to 4MB (#36414) (#36429)
Backport #36414 by @lunny

Fix #36408

Signed-off-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2026-01-21 16:23:00 -08:00
Lunny Xiao
4cdb8a7f96 Fix missing repository id when migrating release attachments (#36389) (#36413)
This PR fixes missed repo_id on the migration of attachments to Gitea.
It also provides a doctor check to fix the dirty data on the database.

Backport #36389
2026-01-20 23:36:45 +02:00
wxiaoguang
38125a8d1d Fix git http service handling (#36396)
Partially backport #36391
2026-01-18 01:42:35 +08:00
Giteabot
175a425825 Fix bug on notification read (#36339) (#36387)
Backport #36339 by @lunny

When a user has been revoked permission to access a repository, the
related notification could still be visited. But the repository's
information should not be leaked any more.

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2026-01-16 09:58:56 +02:00
Lunny Xiao
6132f639e7 Fix bug when compare in the pull request (#36363) (#36372)
The pull request comparison should not use `direct compare`.

Backport #36363
2026-01-15 01:58:42 +00:00
Giteabot
dfe4055b92 Release attachments must belong to the intended repo (#36347) (#36375)
Backport #36347 by @lunny

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2026-01-15 02:30:18 +01:00
Giteabot
5fe9703586 Fix permission check on org project operations (#36318) (#36373)
Backport #36318 by @lunny

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2026-01-14 14:22:27 -08:00
Giteabot
53d67dae28 clean watches when make a repository private and check permission when send release emails (#36319) (#36370)
Backport #36319 by @lunny

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2026-01-14 19:28:02 +01:00
Giteabot
ef6ab681f7 Fix incorrect text content detection (#36364) (#36369)
Backport #36364 by wxiaoguang

Fix #36325

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2026-01-15 00:45:38 +08:00
Giteabot
812a3cffb3 Add more check for stopwatch read or list (#36340) (#36368)
Backport #36340 by @lunny

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2026-01-14 16:04:02 +00:00
Giteabot
669b22100b Fix openid setting check (#36346) (#36361)
Backport #36346 by @lunny

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2026-01-13 16:47:00 +02:00
Giteabot
a0c77673ff fill missing has_code in repository api (#36338) (#36359)
Backport #36338 by @TheFox0x7

fixes: https://github.com/go-gitea/gitea/issues/36332

Co-authored-by: TheFox0x7 <thefox0x7@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2026-01-13 10:31:18 +00:00
Giteabot
d96b68cbf5 Fix notifications pagination query parameters (#36351) (#36358)
Backport #36351 by @wxiaoguang

Fix #36350

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2026-01-13 04:37:20 +00:00
Giteabot
f8ec5b3e43 Fix cancel auto merge bug (#36341) (#36356)
Backport #36341 by @lunny

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2026-01-12 18:55:41 -08:00
Lunny Xiao
11891c2dac Fix delete attachment check (#36320) (#36355)
Backport #36320
2026-01-12 13:48:07 -08:00
Giteabot
2c778ff067 LFS locks must belong to the intended repo (#36344) (#36349)
Backport #36344 by @lunny

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2026-01-12 09:09:16 +02:00
wxiaoguang
83ce45b186 Fix some trivial problems (#36336) (#36337)
Partially backport #36336

1. correctly parse git protocol's "OldCommit NewCommit RefName" line, it
should be explicitly split by space
2. trim space for the "commit status context name" to follow the same
behavior of git_model.NewCommitStatus
2026-01-09 20:59:03 +02:00
silverwind
39e83bd3fd Fix WebAuthn error checking (#36219) (#36235)
Backport of https://github.com/go-gitea/gitea/pull/36219
Fixes: https://github.com/go-gitea/gitea/issues/36216

Now `detectWebAuthnSupport` returns the error type and lets the caller
decide whether they call `webAuthnError` and show the error. It no
longer shows the error during page load when the user has not even
interacted with the feature.

The bug affects all users on HTTP, so I think a quick fix release for
this might be good.
2026-01-06 04:57:59 +00:00
Giteabot
c2f9edd673 fix: prevent panic when GitLab release has more links than sources (#36295) (#36305)
Backport #36295 by argoyle

Fixes #36292

Co-authored-by: Joakim Olsson <joakim@unbound.se>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2026-01-05 17:29:28 +00:00
Giteabot
aa575672ac Fix stats bug when syncing release (#36285) (#36294)
Backport #36285 by @lunny

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2026-01-04 12:14:00 +02:00
TheFox0x7
e9c14723b6 add more routes to the "expensive" list (#36290)
backport: https://github.com/go-gitea/gitea/pull/35547
closes https://github.com/go-gitea/gitea/issues/36229

Signed-off-by: Steven Noonan <steven@uplinklabs.net>
Co-authored-by: Steven Noonan <steven@uplinklabs.net>
2026-01-03 16:59:48 +02:00
Giteabot
76b6e94b5b Always honor user's choice for "delete branch after merge" (#36281) (#36286)
Backport #36281 by wxiaoguang

Fix #36280

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2026-01-03 03:06:10 +00:00
Giteabot
163113d173 Make "commit statuses" API accept slashes in "ref" (#36264) (#36275)
Backport #36264 by wxiaoguang

Fix #36253

Support slashes in `{ref}` (follow GitHub's behavior)

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2026-01-01 03:04:03 +00:00
Lunny Xiao
7d010c6932 Fix panic when get editor config file (#36241) (#36247)
Fix #36239
Backport #36241
2025-12-27 20:29:22 -08:00
Giteabot
b71e688634 Use the requested host for LFS links (#36242) (#36258)
Backport #36242 by sollyu

Co-authored-by: sollyu <43105186+sollyu@users.noreply.github.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-12-27 07:23:18 +00:00
Giteabot
e147a8223a Fix regression in writing authorized principals (#36213) (#36218)
Backport #36213 by peterverraedt

Fixes: #36212

Signed-off-by: Peter Verraedt <peter.verraedt@kuleuven.be>
Co-authored-by: Peter Verraedt <peter.verraedt@kuleuven.be>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-12-20 01:57:26 +00:00
Lunny Xiao
9a7cfd8620 Add changelog for 1.25.3 (#36182) 2025-12-18 09:36:38 -08:00
Lunny Xiao
79f4cd754b Fix bugs when comparing and creating pull request (#36144)
Backport #36166
2025-12-17 16:17:33 -08:00
a1012112796
522cc25921 fix webAuthn insecure error view (#36165) (#36179)
backport #36165

Signed-off-by: a1012112796 <1012112796@qq.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2025-12-17 08:49:29 -08:00
Giteabot
a99ccfdf74 Fix OrgAssignment opts (#36174) (#36177)
Backport #36174 by @wxiaoguang

Fix #36084

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-12-17 18:56:58 +08:00
Giteabot
d448ab9ad4 Check user visibility when redirecting to a renamed user (#36148) (#36159)
Backport #36148 by @lunny

Fix #34169

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2025-12-15 11:42:59 +02:00
wxiaoguang
c97b89a662 Fix bug when viewing the commit diff page with non-ANSI files (#36149) (#36150)
Backport #36149
2025-12-14 01:14:06 +08:00
Giteabot
2dd8ef8368 Fix various bugs (#36139) (#36151)
Backport #36139

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-12-13 13:31:49 +00:00
Giteabot
432e128074 Hide RSS icon when viewing a file not under a branch (#36135) (#36141)
Backport #36135 by @lunny

Fix #35855

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2025-12-12 18:39:16 +01:00
silverwind
8d6442a43e Fix SVG size calulation, only use style attribute (#36133) (#36134)
Backport of https://github.com/go-gitea/gitea/pull/36133, only the
bugfix part.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-12-12 09:40:27 +02:00
wxiaoguang
3d66e75a47 Make Golang correctly delete temp files during uploading (#36128) (#36129)
Fix #36127

Partially backport #36128
And by the way partially backport  #36017
2025-12-11 20:10:59 +01:00
Giteabot
e98d9bb93e Improve math rendering (#36124) (#36125)
Backport #36124 by @wxiaoguang

Fix #36108
Fix #36107

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-12-11 01:40:01 +08:00
Giteabot
a601c09826 Changed a small typo in an error message and code comments. (#36117) (#36123)
Backport #36117 by @schinkelg

Changed a small typo in an English error message and code comments. Very
small PR.

Co-authored-by: Ger Schinkel <schinkelg@users.noreply.github.com>
2025-12-10 18:45:56 +08:00
Giteabot
b5f50ff63b Add strikethrough button to markdown editor (#36087) (#36104)
Backport #36087 by silverwind

Co-authored-by: silverwind <me@silverwind.io>
2025-12-08 09:07:27 +08:00
Giteabot
b1b35e934e Fix the bug when ssh clone with redirect user or repository (#36039) (#36090)
Backport #36039 by @lunny

Fix #36026 

The redirect should be checked when original user/repo doesn't exist.

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2025-12-04 20:06:14 -08:00
Giteabot
544450a212 fix some file icon ui (#36078) (#36088)
Backport #36078 by @a1012112796

fix #36071

looks that's because if an svg in hiden env, it's color added by
`fill="url(#a)"` will become not usefull. by ai helping, I think moving
it out of page by position is a good solution. fell free creat a new
pull request if you have a better soluton. Thanks.
<img width="2198" height="1120" alt="image"
src="https://github.com/user-attachments/assets/bbf7c171-0b7f-412a-a1bc-aea3f1629636"
/>

Signed-off-by: a1012112796 <1012112796@qq.com>
Co-authored-by: a1012112796 <1012112796@qq.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-12-04 19:39:35 +00:00
Giteabot
0ab447005d Use Golang net/smtp instead of gomail's smtp to send email (#36055) (#36083)
Backport #36055 by @lunny

Replace #36032
Fix #36030

This PR use `net/smtp` instead of gomail's smtp. Now
github.com/wneessen/go-mail will be used only for generating email
message body.

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2025-12-04 18:58:01 +00:00
Giteabot
52902d4ece Fix edit user email bug in API (#36068) (#36081)
Backport #36068 by @lunny

Follow #36058 for API edit user bug when editing email.

- The Admin Edit User API includes a breaking change. Previously, when
updating a user with an email from an unallowed domain, the request
would succeed but return a warning in the response headers. Now, the
request will fail and return an error in the response body instead.
- Removed `AdminAddOrSetPrimaryEmailAddress` because it will not be used
any where.

Fix https://github.com/go-gitea/gitea/pull/36058#issuecomment-3600005186

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2025-12-04 18:25:52 +00:00
silverwind
0e91c8a068 Bump toolchain to go1.25.5, misc fixes (#36082)
Backport toolchain change into 1.25. This is needed because of the
govulncheck issue
[present](https://github.com/go-gitea/gitea/actions/runs/19921920886/job/57112316941)
in the branch.

---------

Signed-off-by: silverwind <me@silverwind.io>
2025-12-04 17:57:59 +00:00
Giteabot
45cdc5d8fd Fix bug when updating user email (#36058) (#36066)
Backport #36058 by @lunny

Fix #20390 

We should use `ReplacePrimaryEmailAddress` instead of
`AdminAddOrSetPrimaryEmailAddress` when modify user's email from admin
panel. And also we need a database transaction to keep deletion and
insertion succeed at the same time.

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2025-12-02 02:57:21 +01:00
Zettat123
b276849cd8 Fix Actions pull_request.paths being triggered incorrectly by rebase (#36045) (#36054)
Backport #36045

Partially fix #34710

The bug described in #34710 can be divided into two parts: `push.paths`
and `pull_request.paths`. This PR fixes the issue related to
`pull_request.paths`. The root cause is that the check for whether the
workflow can be triggered happens **before** updating the PR’s merge
base. This causes the file-change detection to use the old merge base.
Therefore, we need to update the merge base first and then check whether
the workflow can be triggered.
2025-11-29 05:45:30 +00:00
Giteabot
46d1d154e8 Fix error handling in mailer and wiki services (#36041) (#36053)
Backport #36041 by @hamkido

- Updated error message in `incoming.go` to remove unnecessary wrapping
of the error.
- Corrected typo in error message in `wiki.go` for clarity.

Co-authored-by: hamkido <hamki.do2000@gmail.com>
2025-11-28 20:34:38 -08:00
Giteabot
f164e38e04 Fix incorrect viewed files counter if file has changed (#36009) (#36047)
Backport #36009 by @bytedream

File changes since last review didn't decrease the viewed files counter

---
<img width="440" height="178" alt="image"
src="https://github.com/user-attachments/assets/da34fcf4-452f-4f71-8da2-97edbfc31fdd"
/>

Also reported here ->
https://github.com/go-gitea/gitea/issues/35803#issuecomment-3567850285

Co-authored-by: bytedream <me@bytedream.dev>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2025-11-28 11:33:37 +01:00
Giteabot
d4d338f1c1 Fix container registry error handling (#36021) (#36037)
Backport #36021 by wxiaoguang

1. the `if` check in `handleCreateManifestResult` didn't handler err
correctly
2. add more error details for debugging

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-11-26 08:01:35 -08:00
Giteabot
f6895f632e Add "site admin" back to profile menu (#36010) (#36013)
Backport #36010 by @wxiaoguang

Fix #35904

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-11-23 14:07:44 -08:00
229 changed files with 4905 additions and 1767 deletions

View File

@@ -11,7 +11,7 @@ jobs:
if: github.repository == 'go-gitea/gitea'
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
- uses: actions/setup-go@v6
with:
go-version-file: go.mod
check-latest: true

View File

@@ -17,7 +17,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
- uses: actions/setup-go@v6
with:
go-version-file: go.mod
check-latest: true
@@ -72,7 +72,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
- uses: actions/setup-go@v6
with:
go-version-file: go.mod
check-latest: true
@@ -84,7 +84,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
- uses: actions/setup-go@v6
with:
go-version-file: go.mod
check-latest: true
@@ -101,7 +101,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
- uses: actions/setup-go@v6
with:
go-version-file: go.mod
check-latest: true
@@ -116,7 +116,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
- uses: actions/setup-go@v6
with:
go-version-file: go.mod
check-latest: true
@@ -145,7 +145,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
- uses: actions/setup-go@v6
with:
go-version-file: go.mod
check-latest: true
@@ -190,7 +190,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
- uses: actions/setup-go@v6
with:
go-version-file: go.mod
check-latest: true

View File

@@ -39,7 +39,7 @@ jobs:
- "9000:9000"
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
- uses: actions/setup-go@v6
with:
go-version-file: go.mod
check-latest: true
@@ -67,7 +67,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
- uses: actions/setup-go@v6
with:
go-version-file: go.mod
check-latest: true
@@ -125,7 +125,7 @@ jobs:
- 10000:10000
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
- uses: actions/setup-go@v6
with:
go-version-file: go.mod
check-latest: true
@@ -178,7 +178,7 @@ jobs:
- "993:993"
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
- uses: actions/setup-go@v6
with:
go-version-file: go.mod
check-latest: true
@@ -218,7 +218,7 @@ jobs:
- 10000:10000
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
- uses: actions/setup-go@v6
with:
go-version-file: go.mod
check-latest: true

View File

@@ -19,7 +19,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
- uses: actions/setup-go@v6
with:
go-version-file: go.mod
check-latest: true

View File

@@ -16,7 +16,7 @@ jobs:
# fetch all commits instead of only the last as some branches are long lived and could have many between versions
# fetch all tags to ensure that "git describe" reports expected Gitea version, eg. v1.21.0-dev-1-g1234567
- run: git fetch --unshallow --quiet --tags --force
- uses: actions/setup-go@v5
- uses: actions/setup-go@v6
with:
go-version-file: go.mod
check-latest: true
@@ -65,7 +65,7 @@ jobs:
# fetch all commits instead of only the last as some branches are long lived and could have many between versions
# fetch all tags to ensure that "git describe" reports expected Gitea version, eg. v1.21.0-dev-1-g1234567
- run: git fetch --unshallow --quiet --tags --force
- uses: actions/setup-go@v5
- uses: actions/setup-go@v6
with:
go-version-file: go.mod
check-latest: true
@@ -107,7 +107,7 @@ jobs:
# fetch all commits instead of only the last as some branches are long lived and could have many between versions
# fetch all tags to ensure that "git describe" reports expected Gitea version, eg. v1.21.0-dev-1-g1234567
- run: git fetch --unshallow --quiet --tags --force
- uses: actions/setup-go@v5
- uses: actions/setup-go@v6
with:
go-version-file: go.mod
check-latest: true

View File

@@ -17,7 +17,7 @@ jobs:
# fetch all commits instead of only the last as some branches are long lived and could have many between versions
# fetch all tags to ensure that "git describe" reports expected Gitea version, eg. v1.21.0-dev-1-g1234567
- run: git fetch --unshallow --quiet --tags --force
- uses: actions/setup-go@v5
- uses: actions/setup-go@v6
with:
go-version-file: go.mod
check-latest: true

View File

@@ -21,7 +21,7 @@ jobs:
# fetch all commits instead of only the last as some branches are long lived and could have many between versions
# fetch all tags to ensure that "git describe" reports expected Gitea version, eg. v1.21.0-dev-1-g1234567
- run: git fetch --unshallow --quiet --tags --force
- uses: actions/setup-go@v5
- uses: actions/setup-go@v6
with:
go-version-file: go.mod
check-latest: true

View File

@@ -4,6 +4,112 @@ This changelog goes through the changes that have been made in each release
without substantial changes to our git log; to see the highlights of what has
been added to each release, please refer to the [blog](https://blog.gitea.com).
## [1.25.5](https://github.com/go-gitea/gitea/releases/tag/1.25.5) - 2026-03-10
* SECURITY
* Toolchain Update to Go 1.25.6 (#36480) (#36487)
* Adjust the toolchain version (#36537) (#36542)
* Update toolchain to 1.25.8 for v1.25 (#36888)
* Prevent redirect bypasses via backslash-encoded paths (#36660) (#36716)
* Fix get release draft permission check (#36659) (#36715)
* Fix a bug user could change another user's primary email (#36586) (#36607)
* Fix OAuth2 authorization code expiry and reuse handling (#36797) (#36851)
* Add validation constraints for repository creation fields (#36671) (#36757)
* Fix bug to check whether user can update pull request branch or rebase branch (#36465) (#36838)
* Add migration http transport for push/sync mirror lfs (#36665) (#36691)
* Fix track time list permission check (#36662) (#36744)
* Fix track time issue id (#36664) (#36689)
* Fix path resolving (#36734) (#36746)
* Fix dump release asset bug (#36799) (#36839)
* Fix org permission API visibility checks for hidden members and private orgs (#36798) (#36841)
* Fix forwarded proto handling for public URL detection (#36810) (#36836)
* Add a git grep search timeout (#36809) (#36835)
* Fix oauth2 s256 (#36462) (#36477)
* ENHANCEMENTS
* Make `security-check` informational only (#36681) (#36852)
* Upgrade to github.com/cloudflare/circl 1.6.3, svgo 4.0.1, markdownlint-cli 0.48.0 (#36840)
* Add some validation on values provided to USER_DISABLED_FEATURES and EXTERNAL_USER_DISABLED_FEATURES (#36688) (#36692)
* Upgrade gogit to 5.16.5 (#36687)
* Add wrap to runner label list (#36565) (#36574)
* Add dnf5 command for Fedora in RPM package instructions (#36527) (#36572)
* Allow scroll propagation outside code editor (#36502) (#36510)
* BUGFIXES
* Fix non-admins unable to automerge PRs from forks (#36833) (#36843)
* Fix bug when pushing mirror with wiki (#36795) (#36807)
* Fix artifacts v4 backend upload problems (#36805) (#36834)
* Fix CRAN package version validation to allow more than 4 version components (#36813) (#36821)
* Fix force push time-line commit comments of pull request (#36653) (#36717)
* Fix SVG height calculation in diff viewer (#36748) (#36750)
* Fix push time bug (#36693) (#36713)
* Fix bug the protected branch rule name is conflicted with renamed branch name (#36650) (#36661)
* Fix bug when do LFS GC (#36500) (#36608)
* Fix focus lost bugs in the Monaco editor (#36609)
* Reprocess htmx content after loading more files (#36568) (#36577)
* Fix assignee sidebar links and empty placeholder (#36559) (#36563)
* Fix issues filter dropdown showing empty label scope section (#36535) (#36544)
* Fix various mermaid bugs (#36547) (#36552)
* Fix data race when uploading container blobs concurrently (#36524) (#36526)
* Correct spacing between username and bot label (#36473) (#36484)
## [1.25.4](https://github.com/go-gitea/gitea/releases/tag/1.25.4) - 2026-01-15
* SECURITY
* Release attachments must belong to the intended repo (#36347) (#36375)
* Fix permission check on org project operations (#36318) (#36373)
* Clean watches when make a repository private and check permission when send release emails (#36319) (#36370)
* Add more check for stopwatch read or list (#36340) (#36368)
* Fix openid setting check (#36346) (#36361)
* Fix cancel auto merge bug (#36341) (#36356)
* Fix delete attachment check (#36320) (#36355)
* LFS locks must belong to the intended repo (#36344) (#36349)
* Fix bug on notification read (#36339) #36387
* ENHANCEMENTS
* Add more routes to the "expensive" list (#36290)
* Make "commit statuses" API accept slashes in "ref" (#36264) (#36275)
* BUGFIXES
* Fix git http service handling #36396
* Fix markdown newline handling during IME composition (#36421) #36424
* Fix missing repository id when migrating release attachments (#36389)
* Fix bug when compare in the pull request (#36363) (#36372)
* Fix incorrect text content detection (#36364) (#36369)
* Fill missing `has_code` in repository api (#36338) (#36359)
* Fix notifications pagination query parameters (#36351) (#36358)
* Fix some trivial problems (#36336) (#36337)
* Prevent panic when GitLab release has more links than sources (#36295) (#36305)
* Fix stats bug when syncing release (#36285) (#36294)
* Always honor user's choice for "delete branch after merge" (#36281) (#36286)
* Use the requested host for LFS links (#36242) (#36258)
* Fix panic when get editor config file (#36241) (#36247)
* Fix regression in writing authorized principals (#36213) (#36218)
* Fix WebAuthn error checking (#36219) (#36235)
## [1.25.3](https://github.com/go-gitea/gitea/releases/tag/1.25.3) - 2025-12-17
* SECURITY
* Bump toolchain to go1.25.5, misc fixes (#36082)
* ENHANCEMENTS
* Add strikethrough button to markdown editor (#36087) (#36104)
* Add "site admin" back to profile menu (#36010) (#36013)
* Improve math rendering (#36124) (#36125)
* BUGFIXES
* Check user visibility when redirecting to a renamed user (#36148) (#36159)
* Fix various bugs (#36139) (#36151)
* Fix bug when viewing the commit diff page with non-ANSI files (#36149) (#36150)
* Hide RSS icon when viewing a file not under a branch (#36135) (#36141)
* Fix SVG size calulation, only use `style` attribute (#36133) (#36134)
* Make Golang correctly delete temp files during uploading (#36128) (#36129)
* Fix the bug when ssh clone with redirect user or repository (#36039) (#36090)
* Use Golang net/smtp instead of gomail's smtp to send email (#36055) (#36083)
* Fix edit user email bug in API (#36068) (#36081)
* Fix bug when updating user email (#36058) (#36066)
* Fix incorrect viewed files counter if file has changed (#36009) (#36047)
* Fix container registry error handling (#36021) (#36037)
* Fix webAuthn insecure error view (#36165) (#36179)
* Fix some file icon ui (#36078) (#36088)
* Fix Actions `pull_request.paths` being triggered incorrectly by rebase (#36045) (#36054)
* Fix error handling in mailer and wiki services (#36041) (#36053)
* Fix bugs when comparing and creating pull request (#36166) (#36144)
## [1.25.2](https://github.com/go-gitea/gitea/releases/tag/1.25.2) - 2025-11-23
* SECURITY

View File

@@ -166,19 +166,19 @@ Here's how to run the test suite:
- code lint
| | |
| :-------------------- | :---------------------------------------------------------------- |
|``make lint`` | lint everything (not needed if you only change the front- **or** backend) |
|``make lint-frontend`` | lint frontend files |
|``make lint-backend`` | lint backend files |
| | |
| :-------------------- | :------------------------------------------------------------------------ |
|``make lint`` | lint everything (not needed if you only change the front- **or** backend) |
|``make lint-frontend`` | lint frontend files |
|``make lint-backend`` | lint backend files |
- run tests (we suggest running them on Linux)
| Command | Action | |
| :------------------------------------- | :----------------------------------------------- | ------------ |
|``make test[\#SpecificTestName]`` | run unit test(s) | |
|``make test-sqlite[\#SpecificTestName]``| run [integration](tests/integration) test(s) for SQLite |[More details](tests/integration/README.md) |
|``make test-e2e-sqlite[\#SpecificTestName]``| run [end-to-end](tests/e2e) test(s) for SQLite |[More details](tests/e2e/README.md) |
| Command | Action | |
| :----------------------------------------- | :------------------------------------------------------- | ------------------------------------------ |
|``make test[\#SpecificTestName]`` | run unit test(s) | |
|``make test-sqlite[\#SpecificTestName]``. | run [integration](tests/integration) test(s) for SQLite |[More details](tests/integration/README.md) |
|``make test-e2e-sqlite[\#SpecificTestName]``| run [end-to-end](tests/e2e) test(s) for SQLite |[More details](tests/e2e/README.md) |
## Translation

View File

@@ -35,7 +35,7 @@ SWAGGER_PACKAGE ?= github.com/go-swagger/go-swagger/cmd/swagger@717e3cb29becaaf0
XGO_PACKAGE ?= src.techknowlogick.com/xgo@latest
GO_LICENSES_PACKAGE ?= github.com/google/go-licenses@v1
GOVULNCHECK_PACKAGE ?= golang.org/x/vuln/cmd/govulncheck@v1
ACTIONLINT_PACKAGE ?= github.com/rhysd/actionlint/cmd/actionlint@v1
ACTIONLINT_PACKAGE ?= github.com/rhysd/actionlint/cmd/actionlint@v1.7.8
GOPLS_PACKAGE ?= golang.org/x/tools/gopls@v0.20.0
GOPLS_MODERNIZE_PACKAGE ?= golang.org/x/tools/gopls/internal/analysis/modernize/cmd/modernize@v0.20.0
@@ -766,7 +766,7 @@ generate-go: $(TAGS_PREREQ)
.PHONY: security-check
security-check:
go run $(GOVULNCHECK_PACKAGE) -show color ./...
go run $(GOVULNCHECK_PACKAGE) -show color ./... || true
$(EXECUTABLE): $(GO_SOURCES) $(TAGS_PREREQ)
ifneq ($(and $(STATIC),$(findstring pam,$(TAGS))),)

View File

@@ -163,6 +163,14 @@ func (n *nilWriter) WriteString(s string) (int, error) {
return len(s), nil
}
func parseGitHookCommitRefLine(line string) (oldCommitID, newCommitID string, refFullName git.RefName, ok bool) {
fields := strings.Split(line, " ")
if len(fields) != 3 {
return "", "", "", false
}
return fields[0], fields[1], git.RefName(fields[2]), true
}
func runHookPreReceive(ctx context.Context, c *cli.Command) error {
if isInternal, _ := strconv.ParseBool(os.Getenv(repo_module.EnvIsInternal)); isInternal {
return nil
@@ -197,6 +205,7 @@ Gitea or set your environment appropriately.`, "")
PullRequestID: prID,
DeployKeyID: deployKeyID,
ActionPerm: int(actionPerm),
IsWiki: isWiki,
}
scanner := bufio.NewScanner(os.Stdin)
@@ -228,14 +237,11 @@ Gitea or set your environment appropriately.`, "")
continue
}
fields := bytes.Fields(scanner.Bytes())
if len(fields) != 3 {
oldCommitID, newCommitID, refFullName, ok := parseGitHookCommitRefLine(scanner.Text())
if !ok {
continue
}
oldCommitID := string(fields[0])
newCommitID := string(fields[1])
refFullName := git.RefName(fields[2])
total++
lastline++
@@ -361,6 +367,7 @@ Gitea or set your environment appropriately.`, "")
GitPushOptions: pushOptions(),
PullRequestID: prID,
PushTrigger: repo_module.PushTrigger(os.Getenv(repo_module.EnvPushTrigger)),
IsWiki: isWiki,
}
oldCommitIDs := make([]string, hookBatchSize)
newCommitIDs := make([]string, hookBatchSize)
@@ -378,16 +385,13 @@ Gitea or set your environment appropriately.`, "")
continue
}
fields := bytes.Fields(scanner.Bytes())
if len(fields) != 3 {
var ok bool
oldCommitIDs[count], newCommitIDs[count], refFullNames[count], ok = parseGitHookCommitRefLine(scanner.Text())
if !ok {
continue
}
fmt.Fprintf(out, ".")
oldCommitIDs[count] = string(fields[0])
newCommitIDs[count] = string(fields[1])
refFullNames[count] = git.RefName(fields[2])
commitID, _ := git.NewIDFromString(newCommitIDs[count])
if refFullNames[count] == git.BranchPrefix+"master" && !commitID.IsZero() && count == total {
masterPushed = true
@@ -511,6 +515,7 @@ Gitea or set your environment appropriately.`, "")
reader := bufio.NewReader(os.Stdin)
repoUser := os.Getenv(repo_module.EnvRepoUsername)
isWiki, _ := strconv.ParseBool(os.Getenv(repo_module.EnvRepoIsWiki))
repoName := os.Getenv(repo_module.EnvRepoName)
pusherID, _ := strconv.ParseInt(os.Getenv(repo_module.EnvPusherID), 10, 64)
pusherName := os.Getenv(repo_module.EnvPusherName)
@@ -588,6 +593,7 @@ Gitea or set your environment appropriately.`, "")
UserName: pusherName,
UserID: pusherID,
GitPushOptions: make(map[string]string),
IsWiki: isWiki,
}
hookOptions.OldCommitIDs = make([]string, 0, hookBatchSize)
hookOptions.NewCommitIDs = make([]string, 0, hookBatchSize)

View File

@@ -39,3 +39,17 @@ func TestPktLine(t *testing.T) {
assert.NoError(t, err)
assert.Equal(t, []byte("0007a\nb"), w.Bytes())
}
func TestParseGitHookCommitRefLine(t *testing.T) {
oldCommitID, newCommitID, refName, ok := parseGitHookCommitRefLine("a b c")
assert.True(t, ok)
assert.Equal(t, "a", oldCommitID)
assert.Equal(t, "b", newCommitID)
assert.Equal(t, "c", string(refName))
_, _, _, ok = parseGitHookCommitRefLine("a\tb\tc")
assert.False(t, ok)
_, _, _, ok = parseGitHookCommitRefLine("a b")
assert.False(t, ok)
}

8
go.mod
View File

@@ -1,6 +1,8 @@
module code.gitea.io/gitea
go 1.25.4
go 1.25.0
toolchain go1.25.8
// rfc5280 said: "The serial number is an integer assigned by the CA to each certificate."
// But some CAs use negative serial number, just relax the check. related:
@@ -56,7 +58,7 @@ require (
github.com/go-co-op/gocron v1.37.0
github.com/go-enry/go-enry/v2 v2.9.2
github.com/go-git/go-billy/v5 v5.6.2
github.com/go-git/go-git/v5 v5.16.3
github.com/go-git/go-git/v5 v5.16.5
github.com/go-ldap/ldap/v3 v3.4.11
github.com/go-redsync/redsync/v4 v4.13.0
github.com/go-sql-driver/mysql v1.9.3
@@ -179,7 +181,7 @@ require (
github.com/caddyserver/zerossl v0.1.3 // indirect
github.com/cention-sany/utf7 v0.0.0-20170124080048-26cad61bd60a // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/cloudflare/circl v1.6.1 // indirect
github.com/cloudflare/circl v1.6.3 // indirect
github.com/couchbase/go-couchbase v0.1.1 // indirect
github.com/couchbase/gomemcached v0.3.3 // indirect
github.com/couchbase/goutils v0.1.2 // indirect

8
go.sum
View File

@@ -231,8 +231,8 @@ github.com/chzyer/readline v1.5.1/go.mod h1:Eh+b79XXUwfKfcPLepksvw2tcLE/Ct21YObk
github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=
github.com/chzyer/test v1.0.0/go.mod h1:2JlltgoNkt4TW/z9V/IzDdFaMTM2JPIi26O1pF38GC8=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cloudflare/circl v1.6.1 h1:zqIqSPIndyBh1bjLVVDHMPpVKqp8Su/V+6MeDzzQBQ0=
github.com/cloudflare/circl v1.6.1/go.mod h1:uddAzsPgqdMAYatqJ0lsjX1oECcQLIlRpzZh3pJrofs=
github.com/cloudflare/circl v1.6.3 h1:9GPOhQGF9MCYUeXyMYlqTR6a5gTrgR/fBLXvUgtVcg8=
github.com/cloudflare/circl v1.6.3/go.mod h1:2eXP6Qfat4O/Yhh8BznvKnJ+uzEoTQ6jVKJRn81BiS4=
github.com/coreos/etcd v3.3.10+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=
github.com/coreos/go-etcd v2.0.0+incompatible/go.mod h1:Jez6KQU2B/sWsbdaef3ED8NzMklzPG4d5KIOhIy30Tk=
github.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
@@ -339,8 +339,8 @@ github.com/go-git/go-billy/v5 v5.6.2 h1:6Q86EsPXMa7c3YZ3aLAQsMA0VlWmy43r6FHqa/UN
github.com/go-git/go-billy/v5 v5.6.2/go.mod h1:rcFC2rAsp/erv7CMz9GczHcuD0D32fWzH+MJAU+jaUU=
github.com/go-git/go-git-fixtures/v4 v4.3.2-0.20231010084843-55a94097c399 h1:eMje31YglSBqCdIqdhKBW8lokaMrL3uTkpGYlE2OOT4=
github.com/go-git/go-git-fixtures/v4 v4.3.2-0.20231010084843-55a94097c399/go.mod h1:1OCfN199q1Jm3HZlxleg+Dw/mwps2Wbk9frAWm+4FII=
github.com/go-git/go-git/v5 v5.16.3 h1:Z8BtvxZ09bYm/yYNgPKCzgWtaRqDTgIKRgIRHBfU6Z8=
github.com/go-git/go-git/v5 v5.16.3/go.mod h1:4Ge4alE/5gPs30F2H1esi2gPd69R0C39lolkucHBOp8=
github.com/go-git/go-git/v5 v5.16.5 h1:mdkuqblwr57kVfXri5TTH+nMFLNUxIj9Z7F5ykFbw5s=
github.com/go-git/go-git/v5 v5.16.5/go.mod h1:QOMLpNf1qxuSY4StA/ArOdfFR2TrKEjJiye2kel2m+M=
github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
github.com/go-ini/ini v1.67.0 h1:z6ZrTEZqSWOTyH2FlglNbNgARyHG8oLW9gMELqKr06A=

View File

@@ -10,6 +10,7 @@ import (
"io"
"os"
"path/filepath"
"regexp"
"strings"
"sync"
@@ -50,12 +51,42 @@ func WriteAuthorizedStringForValidKey(key *PublicKey, w io.Writer) error {
return err
}
var globalVars = sync.OnceValue(func() (ret struct {
principalRegexp *regexp.Regexp
},
) {
// principalRegexp expresses whether a principal is considered valid.
// This reverse engineers how sshd parses the authorized keys file,
// see e.g. https://github.com/openssh/openssh-portable/blob/32deb00b38b4ee2b3302f261ea1e68c04e020a08/auth2-pubkeyfile.c#L221-L256
// Any newline or # comment will be stripped when parsing, so don't allow
// those. Also, if any space or tab is present in the principal, the part
// proceeding this would be parsed as an option, so just avoid any whitespace
// altogether.
ret.principalRegexp = regexp.MustCompile(`^[^\s#]+$`)
return ret
})
func writeAuthorizedStringForKey(key *PublicKey, w io.Writer) (keyValid bool, err error) {
const tpl = AuthorizedStringCommentPrefix + "\n" + `command=%s,no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty,no-user-rc,restrict %s %s` + "\n"
pubKey, _, _, _, err := ssh.ParseAuthorizedKey([]byte(key.Content))
if err != nil {
return false, err
const tpl = AuthorizedStringCommentPrefix + "\n" + `command=%s,no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty,no-user-rc,restrict %s` + "\n"
var sshKey string
if key.Type == KeyTypePrincipal {
// TODO: actually using PublicKey to store "principal" is an abuse
if !globalVars().principalRegexp.MatchString(key.Content) {
return false, fmt.Errorf("invalid principal key: %s", key.Content)
}
sshKey = fmt.Sprintf("%s # user-%d", key.Content, key.OwnerID)
} else {
pubKey, _, _, _, err := ssh.ParseAuthorizedKey([]byte(key.Content))
if err != nil {
return false, err
}
sshKeyMarshalled := strings.TrimSpace(string(ssh.MarshalAuthorizedKey(pubKey)))
sshKey = fmt.Sprintf("%s user-%d", sshKeyMarshalled, key.OwnerID)
}
// now the key is valid, the code below could only return template/IO related errors
sbCmd := &strings.Builder{}
err = setting.SSH.AuthorizedKeysCommandTemplateTemplate.Execute(sbCmd, map[string]any{
@@ -69,9 +100,7 @@ func writeAuthorizedStringForKey(key *PublicKey, w io.Writer) (keyValid bool, er
return true, err
}
sshCommandEscaped := util.ShellEscape(sbCmd.String())
sshKeyMarshalled := strings.TrimSpace(string(ssh.MarshalAuthorizedKey(pubKey)))
sshKeyComment := fmt.Sprintf("user-%d", key.OwnerID)
_, err = fmt.Fprintf(w, tpl, sshCommandEscaped, sshKeyMarshalled, sshKeyComment)
_, err = fmt.Fprintf(w, tpl, sshCommandEscaped, sshKey)
return true, err
}

View File

@@ -0,0 +1,90 @@
// Copyright 2025 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package asymkey
import (
"strings"
"testing"
"code.gitea.io/gitea/modules/setting"
"code.gitea.io/gitea/modules/test"
"github.com/stretchr/testify/assert"
)
func TestWriteAuthorizedStringForKey(t *testing.T) {
defer test.MockVariableValue(&setting.AppPath, "/tmp/gitea")()
defer test.MockVariableValue(&setting.CustomConf, "/tmp/app.ini")()
writeKey := func(t *testing.T, key *PublicKey) (bool, string, error) {
sb := &strings.Builder{}
valid, err := writeAuthorizedStringForKey(key, sb)
return valid, sb.String(), err
}
const validKeyContent = `ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICV0MGX/W9IvLA4FXpIuUcdDcbj5KX4syHgsTy7soVgf`
testValid := func(t *testing.T, key *PublicKey, expected string) {
valid, content, err := writeKey(t, key)
assert.True(t, valid)
assert.Equal(t, expected, content)
assert.NoError(t, err)
}
testInvalid := func(t *testing.T, key *PublicKey) {
valid, content, err := writeKey(t, key)
assert.False(t, valid)
assert.Empty(t, content)
assert.Error(t, err)
}
t.Run("PublicKey", func(t *testing.T) {
testValid(t, &PublicKey{
OwnerID: 123,
Content: validKeyContent + " any-comment",
Type: KeyTypeUser,
}, `# gitea public key
command="/tmp/gitea --config=/tmp/app.ini serv key-0",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty,no-user-rc,restrict ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICV0MGX/W9IvLA4FXpIuUcdDcbj5KX4syHgsTy7soVgf user-123
`)
})
t.Run("PublicKeyWithNewLine", func(t *testing.T) {
testValid(t, &PublicKey{
OwnerID: 123,
Content: validKeyContent + "\nany-more", // the new line should be ignored
Type: KeyTypeUser,
}, `# gitea public key
command="/tmp/gitea --config=/tmp/app.ini serv key-0",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty,no-user-rc,restrict ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICV0MGX/W9IvLA4FXpIuUcdDcbj5KX4syHgsTy7soVgf user-123
`)
})
t.Run("PublicKeyInvalid", func(t *testing.T) {
testInvalid(t, &PublicKey{
OwnerID: 123,
Content: validKeyContent + "any-more",
Type: KeyTypeUser,
})
})
t.Run("Principal", func(t *testing.T) {
testValid(t, &PublicKey{
OwnerID: 123,
Content: "any-content",
Type: KeyTypePrincipal,
}, `# gitea public key
command="/tmp/gitea --config=/tmp/app.ini serv key-0",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty,no-user-rc,restrict any-content # user-123
`)
})
t.Run("PrincipalInvalid", func(t *testing.T) {
testInvalid(t, &PublicKey{
OwnerID: 123,
Content: "a b",
Type: KeyTypePrincipal,
})
testInvalid(t, &PublicKey{
OwnerID: 123,
Content: "a\nb",
Type: KeyTypePrincipal,
})
})
}

View File

@@ -14,6 +14,7 @@ import (
"net/url"
"slices"
"strings"
"time"
"code.gitea.io/gitea/models/db"
"code.gitea.io/gitea/modules/container"
@@ -27,6 +28,11 @@ import (
"xorm.io/xorm"
)
// Authorization codes should expire within 10 minutes per https://datatracker.ietf.org/doc/html/rfc6749#section-4.1.2
const oauth2AuthorizationCodeValidity = 10 * time.Minute
var ErrOAuth2AuthorizationCodeInvalidated = errors.New("oauth2 authorization code already invalidated")
// OAuth2Application represents an OAuth2 client (RFC 6749)
type OAuth2Application struct {
ID int64 `xorm:"pk autoincr"`
@@ -386,6 +392,14 @@ func (code *OAuth2AuthorizationCode) TableName() string {
return "oauth2_authorization_code"
}
// IsExpired reports whether the authorization code is expired.
func (code *OAuth2AuthorizationCode) IsExpired() bool {
if code.ValidUntil.IsZero() {
return true
}
return code.ValidUntil <= timeutil.TimeStampNow()
}
// GenerateRedirectURI generates a redirect URI for a successful authorization request. State will be used if not empty.
func (code *OAuth2AuthorizationCode) GenerateRedirectURI(state string) (*url.URL, error) {
redirect, err := url.Parse(code.RedirectURI)
@@ -403,8 +417,14 @@ func (code *OAuth2AuthorizationCode) GenerateRedirectURI(state string) (*url.URL
// Invalidate deletes the auth code from the database to invalidate this code
func (code *OAuth2AuthorizationCode) Invalidate(ctx context.Context) error {
_, err := db.GetEngine(ctx).ID(code.ID).NoAutoCondition().Delete(code)
return err
affected, err := db.GetEngine(ctx).ID(code.ID).NoAutoCondition().Delete(code)
if err != nil {
return err
}
if affected == 0 {
return ErrOAuth2AuthorizationCodeInvalidated
}
return nil
}
// ValidateCodeChallenge validates the given verifier against the saved code challenge. This is part of the PKCE implementation.
@@ -472,6 +492,7 @@ func (grant *OAuth2Grant) GenerateNewAuthorizationCode(ctx context.Context, redi
// for code scanners to grab sensitive tokens.
codeSecret := "gta_" + base32Lower.EncodeToString(rBytes)
validUntil := time.Now().Add(oauth2AuthorizationCodeValidity)
code = &OAuth2AuthorizationCode{
Grant: grant,
GrantID: grant.ID,
@@ -479,6 +500,7 @@ func (grant *OAuth2Grant) GenerateNewAuthorizationCode(ctx context.Context, redi
Code: codeSecret,
CodeChallenge: codeChallenge,
CodeChallengeMethod: codeChallengeMethod,
ValidUntil: timeutil.TimeStamp(validUntil.Unix()),
}
if err := db.Insert(ctx, code); err != nil {
return nil, err

View File

@@ -5,13 +5,45 @@ package auth_test
import (
"testing"
"time"
auth_model "code.gitea.io/gitea/models/auth"
"code.gitea.io/gitea/models/unittest"
"code.gitea.io/gitea/modules/timeutil"
"github.com/stretchr/testify/assert"
)
func TestOAuth2AuthorizationCodeValidity(t *testing.T) {
assert.NoError(t, unittest.PrepareTestDatabase())
t.Run("GenerateSetsValidUntil", func(t *testing.T) {
grant := unittest.AssertExistsAndLoadBean(t, &auth_model.OAuth2Grant{ID: 1})
expectedValidUntil := timeutil.TimeStamp(time.Now().Unix() + 600)
code, err := grant.GenerateNewAuthorizationCode(t.Context(), "http://127.0.0.1/", "", "")
assert.NoError(t, err)
assert.Equal(t, expectedValidUntil, code.ValidUntil)
assert.False(t, code.IsExpired())
assert.NoError(t, code.Invalidate(t.Context()))
})
t.Run("Expired", func(t *testing.T) {
defer timeutil.MockSet(time.Unix(2, 0).UTC())()
code := &auth_model.OAuth2AuthorizationCode{ValidUntil: timeutil.TimeStamp(1)}
assert.True(t, code.IsExpired())
})
t.Run("InvalidateTwice", func(t *testing.T) {
code, err := auth_model.GetOAuth2AuthorizationByCode(t.Context(), "authcode")
assert.NoError(t, err)
if assert.NotNil(t, code) {
assert.NoError(t, code.Invalidate(t.Context()))
assert.ErrorIs(t, code.Invalidate(t.Context()), auth_model.ErrOAuth2AuthorizationCodeInvalidated)
}
})
}
func TestOAuth2Application_GenerateClientSecret(t *testing.T) {
assert.NoError(t, unittest.PrepareTestDatabase())
app := unittest.AssertExistsAndLoadBean(t, &auth_model.OAuth2Application{ID: 1})

View File

@@ -153,3 +153,16 @@
download_count: 0
size: 0
created_unix: 946684800
-
id: 13
uuid: a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a23
repo_id: 1
issue_id: 0
release_id: 4
uploader_id: 2
comment_id: 0
name: draft-attach
download_count: 0
size: 0
created_unix: 946684800

View File

@@ -397,10 +397,16 @@ func RenameBranch(ctx context.Context, repo *repo_model.Repository, from, to str
if protectedBranch != nil {
// there is a protect rule for this branch
protectedBranch.RuleName = to
if _, err = sess.ID(protectedBranch.ID).Cols("branch_name").Update(protectedBranch); err != nil {
existingRule, err := GetProtectedBranchRuleByName(ctx, repo.ID, to)
if err != nil {
return err
}
if existingRule == nil || existingRule.ID == protectedBranch.ID {
protectedBranch.RuleName = to
if _, err = sess.ID(protectedBranch.ID).Cols("branch_name").Update(protectedBranch); err != nil {
return err
}
}
} else {
// some glob protect rules may match this branch
protected, err := IsBranchProtected(ctx, repo.ID, from)
@@ -444,7 +450,7 @@ func RenameBranch(ctx context.Context, repo *repo_model.Repository, from, to str
type FindRecentlyPushedNewBranchesOptions struct {
Repo *repo_model.Repository
BaseRepo *repo_model.Repository
CommitAfterUnix int64
PushedAfterUnix int64
MaxCount int
}
@@ -454,11 +460,11 @@ type RecentlyPushedNewBranch struct {
BranchDisplayName string
BranchLink string
BranchCompareURL string
CommitTime timeutil.TimeStamp
PushedTime timeutil.TimeStamp
}
// FindRecentlyPushedNewBranches return at most 2 new branches pushed by the user in 2 hours which has no opened PRs created
// if opts.CommitAfterUnix is 0, we will find the branches that were committed to in the last 2 hours
// if opts.PushedAfterUnix is 0, we will find the branches that were pushed in the last 2 hours
// if opts.ListOptions is not set, we will only display top 2 latest branches.
// Protected branches will be skipped since they are unlikely to be used to create new PRs.
func FindRecentlyPushedNewBranches(ctx context.Context, doer *user_model.User, opts FindRecentlyPushedNewBranchesOptions) ([]*RecentlyPushedNewBranch, error) {
@@ -486,8 +492,8 @@ func FindRecentlyPushedNewBranches(ctx context.Context, doer *user_model.User, o
}
repoIDs := builder.Select("id").From("repository").Where(repoCond)
if opts.CommitAfterUnix == 0 {
opts.CommitAfterUnix = time.Now().Add(-time.Hour * 2).Unix()
if opts.PushedAfterUnix == 0 {
opts.PushedAfterUnix = time.Now().Add(-time.Hour * 2).Unix()
}
baseBranch, err := GetBranch(ctx, opts.BaseRepo.ID, opts.BaseRepo.DefaultBranch)
@@ -503,7 +509,7 @@ func FindRecentlyPushedNewBranches(ctx context.Context, doer *user_model.User, o
"pusher_id": doer.ID,
"is_deleted": false,
},
builder.Gte{"commit_time": opts.CommitAfterUnix},
builder.Gte{"updated_unix": opts.PushedAfterUnix},
builder.In("repo_id", repoIDs),
// newly created branch have no changes, so skip them
builder.Neq{"commit_id": baseBranch.CommitID},
@@ -556,7 +562,7 @@ func FindRecentlyPushedNewBranches(ctx context.Context, doer *user_model.User, o
BranchName: branch.Name,
BranchLink: fmt.Sprintf("%s/src/branch/%s", branch.Repo.Link(), util.PathEscapeSegments(branch.Name)),
BranchCompareURL: branch.Repo.ComposeBranchCompareURL(opts.BaseRepo, branch.Name),
CommitTime: branch.CommitTime,
PushedTime: branch.UpdatedUnix,
})
}
if len(newBranches) == opts.MaxCount {

View File

@@ -6,14 +6,17 @@ package git_test
import (
"context"
"testing"
"time"
"code.gitea.io/gitea/models/db"
git_model "code.gitea.io/gitea/models/git"
issues_model "code.gitea.io/gitea/models/issues"
repo_model "code.gitea.io/gitea/models/repo"
"code.gitea.io/gitea/models/unittest"
user_model "code.gitea.io/gitea/models/user"
"code.gitea.io/gitea/modules/git"
"code.gitea.io/gitea/modules/optional"
"code.gitea.io/gitea/modules/timeutil"
"github.com/stretchr/testify/assert"
)
@@ -63,6 +66,36 @@ func TestGetDeletedBranch(t *testing.T) {
assert.NotNil(t, getDeletedBranch(t, firstBranch))
}
func TestFindRecentlyPushedNewBranchesUsesPushTime(t *testing.T) {
assert.NoError(t, unittest.PrepareTestDatabase())
repo := unittest.AssertExistsAndLoadBean(t, &repo_model.Repository{ID: 10})
doer := unittest.AssertExistsAndLoadBean(t, &user_model.User{ID: 12})
branch := unittest.AssertExistsAndLoadBean(t, &git_model.Branch{RepoID: repo.ID, Name: "outdated-new-branch"})
commitUnix := time.Now().Add(-3 * time.Hour).Unix()
pushUnix := time.Now().Add(-30 * time.Minute).Unix()
_, err := db.GetEngine(t.Context()).Exec(
"UPDATE branch SET commit_time = ?, updated_unix = ? WHERE id = ?",
commitUnix,
pushUnix,
branch.ID,
)
assert.NoError(t, err)
branches, err := git_model.FindRecentlyPushedNewBranches(t.Context(), doer, git_model.FindRecentlyPushedNewBranchesOptions{
Repo: repo,
BaseRepo: repo,
PushedAfterUnix: time.Now().Add(-time.Hour).Unix(),
MaxCount: 1,
})
assert.NoError(t, err)
if assert.Len(t, branches, 1) {
assert.Equal(t, branch.Name, branches[0].BranchName)
assert.Equal(t, timeutil.TimeStamp(pushUnix), branches[0].PushedTime)
}
}
func TestDeletedBranchLoadUser(t *testing.T) {
assert.NoError(t, unittest.PrepareTestDatabase())
@@ -159,6 +192,53 @@ func TestRenameBranch(t *testing.T) {
})
}
func TestRenameBranchProtectedRuleConflict(t *testing.T) {
assert.NoError(t, unittest.PrepareTestDatabase())
repo1 := unittest.AssertExistsAndLoadBean(t, &repo_model.Repository{ID: 1})
master := unittest.AssertExistsAndLoadBean(t, &git_model.Branch{RepoID: repo1.ID, Name: "master"})
devBranch := &git_model.Branch{
RepoID: repo1.ID,
Name: "dev",
CommitID: master.CommitID,
CommitMessage: master.CommitMessage,
CommitTime: master.CommitTime,
PusherID: master.PusherID,
}
assert.NoError(t, db.Insert(t.Context(), devBranch))
pbDev := git_model.ProtectedBranch{
RepoID: repo1.ID,
RuleName: "dev",
CanPush: true,
}
assert.NoError(t, git_model.UpdateProtectBranch(t.Context(), repo1, &pbDev, git_model.WhitelistOptions{}))
pbMain := git_model.ProtectedBranch{
RepoID: repo1.ID,
RuleName: "main",
CanPush: true,
}
assert.NoError(t, git_model.UpdateProtectBranch(t.Context(), repo1, &pbMain, git_model.WhitelistOptions{}))
assert.NoError(t, git_model.RenameBranch(t.Context(), repo1, "dev", "main", func(ctx context.Context, isDefault bool) error {
return nil
}))
unittest.AssertNotExistsBean(t, &git_model.Branch{RepoID: repo1.ID, Name: "dev"})
unittest.AssertExistsAndLoadBean(t, &git_model.Branch{RepoID: repo1.ID, Name: "main"})
protectedDev, err := git_model.GetProtectedBranchRuleByName(t.Context(), repo1.ID, "dev")
assert.NoError(t, err)
assert.NotNil(t, protectedDev)
assert.Equal(t, "dev", protectedDev.RuleName)
protectedMainByID, err := git_model.GetProtectedBranchRuleByID(t.Context(), repo1.ID, pbMain.ID)
assert.NoError(t, err)
assert.NotNil(t, protectedMainByID)
assert.Equal(t, "main", protectedMainByID.RuleName)
}
func TestOnlyGetDeletedBranchOnCorrectRepo(t *testing.T) {
assert.NoError(t, unittest.PrepareTestDatabase())

View File

@@ -343,15 +343,12 @@ func IterateRepositoryIDsWithLFSMetaObjects(ctx context.Context, f func(ctx cont
// IterateLFSMetaObjectsForRepoOptions provides options for IterateLFSMetaObjectsForRepo
type IterateLFSMetaObjectsForRepoOptions struct {
OlderThan timeutil.TimeStamp
UpdatedLessRecentlyThan timeutil.TimeStamp
OrderByUpdated bool
LoopFunctionAlwaysUpdates bool
OlderThan timeutil.TimeStamp
UpdatedLessRecentlyThan timeutil.TimeStamp
}
// IterateLFSMetaObjectsForRepo provides a iterator for LFSMetaObjects per Repo
func IterateLFSMetaObjectsForRepo(ctx context.Context, repoID int64, f func(context.Context, *LFSMetaObject, int64) error, opts *IterateLFSMetaObjectsForRepoOptions) error {
var start int
batchSize := setting.Database.IterateBufferSize
engine := db.GetEngine(ctx)
type CountLFSMetaObject struct {
@@ -359,7 +356,7 @@ func IterateLFSMetaObjectsForRepo(ctx context.Context, repoID int64, f func(cont
LFSMetaObject `xorm:"extends"`
}
id := int64(0)
lastID := int64(0)
for {
beans := make([]*CountLFSMetaObject, 0, batchSize)
@@ -372,29 +369,23 @@ func IterateLFSMetaObjectsForRepo(ctx context.Context, repoID int64, f func(cont
if !opts.UpdatedLessRecentlyThan.IsZero() {
sess.And("`lfs_meta_object`.updated_unix < ?", opts.UpdatedLessRecentlyThan)
}
sess.GroupBy("`lfs_meta_object`.id")
if opts.OrderByUpdated {
sess.OrderBy("`lfs_meta_object`.updated_unix ASC")
} else {
sess.And("`lfs_meta_object`.id > ?", id)
sess.OrderBy("`lfs_meta_object`.id ASC")
}
if err := sess.Limit(batchSize, start).Find(&beans); err != nil {
sess.GroupBy("`lfs_meta_object`.id").
And("`lfs_meta_object`.id > ?", lastID).
OrderBy("`lfs_meta_object`.id ASC")
if err := sess.Limit(batchSize).Find(&beans); err != nil {
return err
}
if len(beans) == 0 {
return nil
}
if !opts.LoopFunctionAlwaysUpdates {
start += len(beans)
}
for _, bean := range beans {
if err := f(ctx, &bean.LFSMetaObject, bean.Count); err != nil {
return err
}
}
id = beans[len(beans)-1].ID
lastID = beans[len(beans)-1].ID
}
}

View File

@@ -108,10 +108,10 @@ func GetLFSLock(ctx context.Context, repo *repo_model.Repository, path string) (
return rel, nil
}
// GetLFSLockByID returns release by given id.
func GetLFSLockByID(ctx context.Context, id int64) (*LFSLock, error) {
// GetLFSLockByIDAndRepo returns lfs lock by given id and repository id.
func GetLFSLockByIDAndRepo(ctx context.Context, id, repoID int64) (*LFSLock, error) {
lock := new(LFSLock)
has, err := db.GetEngine(ctx).ID(id).Get(lock)
has, err := db.GetEngine(ctx).ID(id).And("repo_id = ?", repoID).Get(lock)
if err != nil {
return nil, err
} else if !has {
@@ -160,7 +160,7 @@ func CountLFSLockByRepoID(ctx context.Context, repoID int64) (int64, error) {
// DeleteLFSLockByID deletes a lock by given ID.
func DeleteLFSLockByID(ctx context.Context, id int64, repo *repo_model.Repository, u *user_model.User, force bool) (*LFSLock, error) {
return db.WithTx2(ctx, func(ctx context.Context) (*LFSLock, error) {
lock, err := GetLFSLockByID(ctx, id)
lock, err := GetLFSLockByIDAndRepo(ctx, id, repo.ID)
if err != nil {
return nil, err
}

View File

@@ -0,0 +1,82 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package git
import (
"fmt"
"testing"
"time"
repo_model "code.gitea.io/gitea/models/repo"
"code.gitea.io/gitea/models/unittest"
user_model "code.gitea.io/gitea/models/user"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func createTestLock(t *testing.T, repo *repo_model.Repository, owner *user_model.User) *LFSLock {
t.Helper()
path := fmt.Sprintf("%s-%d-%d", t.Name(), repo.ID, time.Now().UnixNano())
lock, err := CreateLFSLock(t.Context(), repo, &LFSLock{
OwnerID: owner.ID,
Path: path,
})
require.NoError(t, err)
return lock
}
func TestGetLFSLockByIDAndRepo(t *testing.T) {
require.NoError(t, unittest.PrepareTestDatabase())
repo1 := unittest.AssertExistsAndLoadBean(t, &repo_model.Repository{ID: 1})
repo3 := unittest.AssertExistsAndLoadBean(t, &repo_model.Repository{ID: 3})
user2 := unittest.AssertExistsAndLoadBean(t, &user_model.User{ID: 2})
user4 := unittest.AssertExistsAndLoadBean(t, &user_model.User{ID: 4})
lockRepo1 := createTestLock(t, repo1, user2)
lockRepo3 := createTestLock(t, repo3, user4)
fetched, err := GetLFSLockByIDAndRepo(t.Context(), lockRepo1.ID, repo1.ID)
require.NoError(t, err)
assert.Equal(t, lockRepo1.ID, fetched.ID)
assert.Equal(t, repo1.ID, fetched.RepoID)
_, err = GetLFSLockByIDAndRepo(t.Context(), lockRepo1.ID, repo3.ID)
assert.Error(t, err)
assert.True(t, IsErrLFSLockNotExist(err))
_, err = GetLFSLockByIDAndRepo(t.Context(), lockRepo3.ID, repo1.ID)
assert.Error(t, err)
assert.True(t, IsErrLFSLockNotExist(err))
}
func TestDeleteLFSLockByIDRequiresRepoMatch(t *testing.T) {
require.NoError(t, unittest.PrepareTestDatabase())
repo1 := unittest.AssertExistsAndLoadBean(t, &repo_model.Repository{ID: 1})
repo3 := unittest.AssertExistsAndLoadBean(t, &repo_model.Repository{ID: 3})
user2 := unittest.AssertExistsAndLoadBean(t, &user_model.User{ID: 2})
user4 := unittest.AssertExistsAndLoadBean(t, &user_model.User{ID: 4})
lockRepo1 := createTestLock(t, repo1, user2)
lockRepo3 := createTestLock(t, repo3, user4)
_, err := DeleteLFSLockByID(t.Context(), lockRepo3.ID, repo1, user2, true)
assert.Error(t, err)
assert.True(t, IsErrLFSLockNotExist(err))
existing, err := GetLFSLockByIDAndRepo(t.Context(), lockRepo3.ID, repo3.ID)
require.NoError(t, err)
assert.Equal(t, lockRepo3.ID, existing.ID)
deleted, err := DeleteLFSLockByID(t.Context(), lockRepo3.ID, repo3, user4, true)
require.NoError(t, err)
assert.Equal(t, lockRepo3.ID, deleted.ID)
deleted, err = DeleteLFSLockByID(t.Context(), lockRepo1.ID, repo1, user2, false)
require.NoError(t, err)
assert.Equal(t, lockRepo1.ID, deleted.ID)
}

61
models/git/lfs_test.go Normal file
View File

@@ -0,0 +1,61 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package git_test
import (
"bytes"
"context"
"strconv"
"testing"
"time"
"code.gitea.io/gitea/models/db"
git_model "code.gitea.io/gitea/models/git"
repo_model "code.gitea.io/gitea/models/repo"
"code.gitea.io/gitea/models/unittest"
"code.gitea.io/gitea/modules/lfs"
"code.gitea.io/gitea/modules/setting"
"code.gitea.io/gitea/modules/test"
"code.gitea.io/gitea/modules/timeutil"
"github.com/stretchr/testify/assert"
)
func TestIterateLFSMetaObjectsForRepoUpdatesDoNotSkip(t *testing.T) {
assert.NoError(t, unittest.PrepareTestDatabase())
ctx := t.Context()
repo, err := repo_model.GetRepositoryByOwnerAndName(ctx, "user2", "repo1")
assert.NoError(t, err)
defer test.MockVariableValue(&setting.Database.IterateBufferSize, 1)()
created := make([]*git_model.LFSMetaObject, 0, 3)
for i := range 3 {
content := []byte("gitea-lfs-" + strconv.Itoa(i))
pointer, err := lfs.GeneratePointer(bytes.NewReader(content))
assert.NoError(t, err)
meta, err := git_model.NewLFSMetaObject(ctx, repo.ID, pointer)
assert.NoError(t, err)
created = append(created, meta)
}
iterated := make([]int64, 0, len(created))
cutoff := time.Now().Add(24 * time.Hour)
iterErr := git_model.IterateLFSMetaObjectsForRepo(ctx, repo.ID, func(ctx context.Context, meta *git_model.LFSMetaObject, count int64) error {
iterated = append(iterated, meta.ID)
_, err := db.GetEngine(ctx).ID(meta.ID).Cols("updated_unix").Update(&git_model.LFSMetaObject{
UpdatedUnix: timeutil.TimeStamp(time.Now().Unix()),
})
return err
}, &git_model.IterateLFSMetaObjectsForRepoOptions{
OlderThan: timeutil.TimeStamp(cutoff.Unix()),
UpdatedLessRecentlyThan: timeutil.TimeStamp(cutoff.Unix()),
})
assert.NoError(t, iterErr)
expected := []int64{created[0].ID, created[1].ID, created[2].ID}
assert.Equal(t, expected, iterated)
}

View File

@@ -692,7 +692,7 @@ func (c *Comment) LoadTime(ctx context.Context) error {
return nil
}
var err error
c.Time, err = GetTrackedTimeByID(ctx, c.TimeID)
c.Time, err = GetTrackedTimeByID(ctx, c.IssueID, c.TimeID)
return err
}

View File

@@ -12,6 +12,8 @@ import (
user_model "code.gitea.io/gitea/models/user"
"code.gitea.io/gitea/modules/timeutil"
"code.gitea.io/gitea/modules/util"
"xorm.io/builder"
)
// Stopwatch represents a stopwatch for time tracking.
@@ -232,3 +234,14 @@ func CancelStopwatch(ctx context.Context, user *user_model.User, issue *Issue) (
})
return ok, err
}
// RemoveStopwatchesByRepoID removes all stopwatches for a user in a specific repository
// this function should be called before removing all the issues of the repository
func RemoveStopwatchesByRepoID(ctx context.Context, userID, repoID int64) error {
_, err := db.GetEngine(ctx).
Where("`stopwatch`.user_id = ?", userID).
And(builder.In("`stopwatch`.issue_id",
builder.Select("id").From("issue").Where(builder.Eq{"repo_id": repoID}))).
Delete(new(Stopwatch))
return err
}

View File

@@ -311,13 +311,13 @@ func deleteTime(ctx context.Context, t *TrackedTime) error {
}
// GetTrackedTimeByID returns raw TrackedTime without loading attributes by id
func GetTrackedTimeByID(ctx context.Context, id int64) (*TrackedTime, error) {
func GetTrackedTimeByID(ctx context.Context, issueID, trackedTimeID int64) (*TrackedTime, error) {
time := new(TrackedTime)
has, err := db.GetEngine(ctx).ID(id).Get(time)
has, err := db.GetEngine(ctx).ID(trackedTimeID).Where("issue_id = ?", issueID).Get(time)
if err != nil {
return nil, err
} else if !has {
return nil, db.ErrNotExist{Resource: "tracked_time", ID: id}
return nil, db.ErrNotExist{Resource: "tracked_time", ID: trackedTimeID}
}
return time, nil
}

View File

@@ -43,13 +43,15 @@ func GetOrInsertBlob(ctx context.Context, pb *PackageBlob) (*PackageBlob, bool,
existing := &PackageBlob{}
has, err := e.Where(builder.Eq{
hashCond := builder.Eq{
"size": pb.Size,
"hash_md5": pb.HashMD5,
"hash_sha1": pb.HashSHA1,
"hash_sha256": pb.HashSHA256,
"hash_sha512": pb.HashSHA512,
}).Get(existing)
}
has, err := e.Where(hashCond).Get(existing)
if err != nil {
return nil, false, err
}
@@ -57,6 +59,11 @@ func GetOrInsertBlob(ctx context.Context, pb *PackageBlob) (*PackageBlob, bool,
return existing, true, nil
}
if _, err = e.Insert(pb); err != nil {
// Handle race condition: another request may have inserted the same blob
// between our SELECT and INSERT. Retry the SELECT to get the existing blob.
if has, _ = e.Where(hashCond).Get(existing); has {
return existing, true, nil
}
return nil, false, err
}
return pb, false, nil

View File

@@ -0,0 +1,51 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package packages
import (
"testing"
"code.gitea.io/gitea/models/unittest"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"golang.org/x/sync/errgroup"
)
func TestGetOrInsertBlobConcurrent(t *testing.T) {
require.NoError(t, unittest.PrepareTestDatabase())
testBlob := PackageBlob{
Size: 123,
HashMD5: "md5",
HashSHA1: "sha1",
HashSHA256: "sha256",
HashSHA512: "sha512",
}
const numGoroutines = 3
var wg errgroup.Group
results := make([]*PackageBlob, numGoroutines)
existed := make([]bool, numGoroutines)
for idx := range numGoroutines {
wg.Go(func() error {
blob := testBlob // Create a copy of the test blob for each goroutine
var err error
results[idx], existed[idx], err = GetOrInsertBlob(t.Context(), &blob)
return err
})
}
require.NoError(t, wg.Wait())
// then: all GetOrInsertBlob succeeds with the same blob ID, and only one indicates it did not exist before
existedCount := 0
assert.NotNil(t, results[0])
for i := range numGoroutines {
assert.Equal(t, results[0].ID, results[i].ID)
if existed[i] {
existedCount++
}
}
assert.Equal(t, numGoroutines-1, existedCount)
}

View File

@@ -213,6 +213,18 @@ func GetColumn(ctx context.Context, columnID int64) (*Column, error) {
return column, nil
}
func GetColumnByIDAndProjectID(ctx context.Context, columnID, projectID int64) (*Column, error) {
column := new(Column)
has, err := db.GetEngine(ctx).ID(columnID).And("project_id=?", projectID).Get(column)
if err != nil {
return nil, err
} else if !has {
return nil, ErrProjectColumnNotExist{ColumnID: columnID}
}
return column, nil
}
// UpdateColumn updates a project column
func UpdateColumn(ctx context.Context, column *Column) error {
var fieldToUpdate []string

View File

@@ -302,6 +302,18 @@ func GetProjectByID(ctx context.Context, id int64) (*Project, error) {
return p, nil
}
func GetProjectByIDAndOwner(ctx context.Context, id, ownerID int64) (*Project, error) {
p := new(Project)
has, err := db.GetEngine(ctx).ID(id).And("owner_id = ?", ownerID).Get(p)
if err != nil {
return nil, err
} else if !has {
return nil, ErrProjectNotExist{ID: id}
}
return p, nil
}
// GetProjectForRepoByID returns the projects in a repository
func GetProjectForRepoByID(ctx context.Context, repoID, id int64) (*Project, error) {
p := new(Project)

View File

@@ -73,18 +73,18 @@ func GetReviewState(ctx context.Context, userID, pullID int64, commitSHA string)
// UpdateReviewState updates the given review inside the database, regardless of whether it existed before or not
// The given map of files with their viewed state will be merged with the previous review, if present
func UpdateReviewState(ctx context.Context, userID, pullID int64, commitSHA string, updatedFiles map[string]ViewedState) error {
func UpdateReviewState(ctx context.Context, userID, pullID int64, commitSHA string, updatedFiles map[string]ViewedState) (*ReviewState, error) {
log.Trace("Updating review for user %d, repo %d, commit %s with the updated files %v.", userID, pullID, commitSHA, updatedFiles)
review, exists, err := GetReviewState(ctx, userID, pullID, commitSHA)
if err != nil {
return err
return nil, err
}
if exists {
review.UpdatedFiles = mergeFiles(review.UpdatedFiles, updatedFiles)
} else if previousReview, err := getNewestReviewStateApartFrom(ctx, userID, pullID, commitSHA); err != nil {
return err
return nil, err
// Overwrite the viewed files of the previous review if present
} else if previousReview != nil {
@@ -98,11 +98,11 @@ func UpdateReviewState(ctx context.Context, userID, pullID int64, commitSHA stri
if !exists {
log.Trace("Inserting new review for user %d, repo %d, commit %s with the updated files %v.", userID, pullID, commitSHA, review.UpdatedFiles)
_, err := engine.Insert(review)
return err
return nil, err
}
log.Trace("Updating already existing review with ID %d (user %d, repo %d, commit %s) with the updated files %v.", review.ID, userID, pullID, commitSHA, review.UpdatedFiles)
_, err = engine.ID(review.ID).Update(&ReviewState{UpdatedFiles: review.UpdatedFiles})
return err
_, err = engine.ID(review.ID).Cols("updated_files").Update(review)
return review, err
}
// mergeFiles merges the given maps of files with their viewing state into one map.

View File

@@ -166,6 +166,11 @@ func GetAttachmentByReleaseIDFileName(ctx context.Context, releaseID int64, file
return attach, nil
}
func GetUnlinkedAttachmentsByUserID(ctx context.Context, userID int64) ([]*Attachment, error) {
attachments := make([]*Attachment, 0, 10)
return attachments, db.GetEngine(ctx).Where("uploader_id = ? AND issue_id = 0 AND release_id = 0 AND comment_id = 0", userID).Find(&attachments)
}
// DeleteAttachment deletes the given attachment and optionally the associated file.
func DeleteAttachment(ctx context.Context, a *Attachment, remove bool) error {
_, err := DeleteAttachments(ctx, []*Attachment{a}, remove)

View File

@@ -101,3 +101,19 @@ func TestGetAttachmentsByUUIDs(t *testing.T) {
assert.Equal(t, int64(1), attachList[0].IssueID)
assert.Equal(t, int64(5), attachList[1].IssueID)
}
func TestGetUnlinkedAttachmentsByUserID(t *testing.T) {
assert.NoError(t, unittest.PrepareTestDatabase())
attachments, err := repo_model.GetUnlinkedAttachmentsByUserID(t.Context(), 8)
assert.NoError(t, err)
assert.Len(t, attachments, 1)
assert.Equal(t, int64(10), attachments[0].ID)
assert.Zero(t, attachments[0].IssueID)
assert.Zero(t, attachments[0].ReleaseID)
assert.Zero(t, attachments[0].CommentID)
attachments, err = repo_model.GetUnlinkedAttachmentsByUserID(t.Context(), 1)
assert.NoError(t, err)
assert.Empty(t, attachments)
}

View File

@@ -93,15 +93,25 @@ func init() {
db.RegisterModel(new(Release))
}
// LoadAttributes load repo and publisher attributes for a release
func (r *Release) LoadAttributes(ctx context.Context) error {
var err error
if r.Repo == nil {
r.Repo, err = GetRepositoryByID(ctx, r.RepoID)
if err != nil {
return err
}
// LegacyAttachmentMissingRepoIDCutoff marks the date when repo_id started to be written during uploads
// (2026-01-16T00:00:00Z). Older rows might have repo_id=0 and should be tolerated once.
const LegacyAttachmentMissingRepoIDCutoff timeutil.TimeStamp = 1768521600
func (r *Release) LoadRepo(ctx context.Context) (err error) {
if r.Repo != nil {
return nil
}
r.Repo, err = GetRepositoryByID(ctx, r.RepoID)
return err
}
// LoadAttributes load repo and publisher attributes for a release
func (r *Release) LoadAttributes(ctx context.Context) (err error) {
if err := r.LoadRepo(ctx); err != nil {
return err
}
if r.Publisher == nil {
r.Publisher, err = user_model.GetUserByID(ctx, r.PublisherID)
if err != nil {
@@ -168,6 +178,11 @@ func UpdateReleaseNumCommits(ctx context.Context, rel *Release) error {
// AddReleaseAttachments adds a release attachments
func AddReleaseAttachments(ctx context.Context, releaseID int64, attachmentUUIDs []string) (err error) {
rel, err := GetReleaseByID(ctx, releaseID)
if err != nil {
return err
}
// Check attachments
attachments, err := GetAttachmentsByUUIDs(ctx, attachmentUUIDs)
if err != nil {
@@ -175,6 +190,17 @@ func AddReleaseAttachments(ctx context.Context, releaseID int64, attachmentUUIDs
}
for i := range attachments {
if attachments[i].RepoID == 0 && attachments[i].CreatedUnix < LegacyAttachmentMissingRepoIDCutoff {
attachments[i].RepoID = rel.RepoID
if _, err = db.GetEngine(ctx).ID(attachments[i].ID).Cols("repo_id").Update(attachments[i]); err != nil {
return fmt.Errorf("update attachment repo_id [%d]: %w", attachments[i].ID, err)
}
}
if attachments[i].RepoID != rel.RepoID {
return util.NewPermissionDeniedErrorf("attachment belongs to different repository")
}
if attachments[i].ReleaseID != 0 {
return util.NewPermissionDeniedErrorf("release permission denied")
}

View File

@@ -6,7 +6,9 @@ package repo
import (
"testing"
"code.gitea.io/gitea/models/db"
"code.gitea.io/gitea/models/unittest"
"code.gitea.io/gitea/modules/util"
"github.com/stretchr/testify/assert"
)
@@ -37,3 +39,54 @@ func Test_FindTagsByCommitIDs(t *testing.T) {
assert.Equal(t, "delete-tag", rels[1].TagName)
assert.Equal(t, "v1.0", rels[2].TagName)
}
func TestAddReleaseAttachmentsRejectsDifferentRepo(t *testing.T) {
assert.NoError(t, unittest.PrepareTestDatabase())
uuid := "a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a12" // attachment 2 belongs to repo 2
err := AddReleaseAttachments(t.Context(), 1, []string{uuid})
assert.Error(t, err)
assert.ErrorIs(t, err, util.ErrPermissionDenied)
attach, err := GetAttachmentByUUID(t.Context(), uuid)
assert.NoError(t, err)
assert.Zero(t, attach.ReleaseID, "attachment should not be linked to release on failure")
}
func TestAddReleaseAttachmentsAllowsLegacyMissingRepoID(t *testing.T) {
assert.NoError(t, unittest.PrepareTestDatabase())
legacyUUID := "a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a20" // attachment 10 has repo_id 0
err := AddReleaseAttachments(t.Context(), 1, []string{legacyUUID})
assert.NoError(t, err)
attach, err := GetAttachmentByUUID(t.Context(), legacyUUID)
assert.NoError(t, err)
assert.EqualValues(t, 1, attach.RepoID)
assert.EqualValues(t, 1, attach.ReleaseID)
}
func TestAddReleaseAttachmentsRejectsRecentZeroRepoID(t *testing.T) {
assert.NoError(t, unittest.PrepareTestDatabase())
recentUUID := "a0eebc99-9c0b-4ef8-bb6d-6bb9bd3800aa"
attachment := &Attachment{
UUID: recentUUID,
RepoID: 0,
IssueID: 0,
ReleaseID: 0,
CommentID: 0,
Name: "recent-zero",
CreatedUnix: LegacyAttachmentMissingRepoIDCutoff + 1,
}
assert.NoError(t, db.Insert(t.Context(), attachment))
err := AddReleaseAttachments(t.Context(), 1, []string{recentUUID})
assert.Error(t, err)
assert.ErrorIs(t, err, util.ErrPermissionDenied)
attach, err := GetAttachmentByUUID(t.Context(), recentUUID)
assert.NoError(t, err)
assert.Zero(t, attach.ReleaseID)
assert.Zero(t, attach.RepoID)
}

View File

@@ -176,3 +176,13 @@ func WatchIfAuto(ctx context.Context, userID, repoID int64, isWrite bool) error
}
return watchRepoMode(ctx, watch, WatchModeAuto)
}
// ClearRepoWatches clears all watches for a repository and from the user that watched it.
// Used when a repository is set to private.
func ClearRepoWatches(ctx context.Context, repoID int64) error {
if _, err := db.Exec(ctx, "UPDATE `repository` SET num_watches = 0 WHERE id = ?", repoID); err != nil {
return err
}
return db.DeleteBeans(ctx, Watch{RepoID: repoID})
}

View File

@@ -13,6 +13,7 @@ import (
"code.gitea.io/gitea/modules/setting"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestIsWatching(t *testing.T) {
@@ -119,3 +120,21 @@ func TestWatchIfAuto(t *testing.T) {
assert.NoError(t, err)
assert.Len(t, watchers, prevCount)
}
func TestClearRepoWatches(t *testing.T) {
assert.NoError(t, unittest.PrepareTestDatabase())
const repoID int64 = 1
watchers, err := repo_model.GetRepoWatchersIDs(t.Context(), repoID)
require.NoError(t, err)
require.NotEmpty(t, watchers)
assert.NoError(t, repo_model.ClearRepoWatches(t.Context(), repoID))
watchers, err = repo_model.GetRepoWatchersIDs(t.Context(), repoID)
assert.NoError(t, err)
assert.Empty(t, watchers)
repo := unittest.AssertExistsAndLoadBean(t, &repo_model.Repository{ID: repoID})
assert.Zero(t, repo.NumWatches)
}

View File

@@ -276,17 +276,22 @@ func updateActivation(ctx context.Context, email *EmailAddress, activate bool) e
return UpdateUserCols(ctx, user, "rands")
}
func MakeActiveEmailPrimary(ctx context.Context, emailID int64) error {
return makeEmailPrimaryInternal(ctx, emailID, true)
func MakeActiveEmailPrimary(ctx context.Context, ownerID, emailID int64) error {
return makeEmailPrimaryInternal(ctx, ownerID, emailID, true)
}
func MakeInactiveEmailPrimary(ctx context.Context, emailID int64) error {
return makeEmailPrimaryInternal(ctx, emailID, false)
func MakeInactiveEmailPrimary(ctx context.Context, ownerID, emailID int64) error {
return makeEmailPrimaryInternal(ctx, ownerID, emailID, false)
}
func makeEmailPrimaryInternal(ctx context.Context, emailID int64, isActive bool) error {
func makeEmailPrimaryInternal(ctx context.Context, ownerID, emailID int64, isActive bool) error {
email := &EmailAddress{}
if has, err := db.GetEngine(ctx).ID(emailID).Where(builder.Eq{"is_activated": isActive}).Get(email); err != nil {
if has, err := db.GetEngine(ctx).ID(emailID).
Where(builder.Eq{
"uid": ownerID,
"is_activated": isActive,
}).
Get(email); err != nil {
return err
} else if !has {
return ErrEmailAddressNotExist{}
@@ -336,7 +341,7 @@ func ChangeInactivePrimaryEmail(ctx context.Context, uid int64, oldEmailAddr, ne
if err != nil {
return err
}
return MakeInactiveEmailPrimary(ctx, newEmail.ID)
return MakeInactiveEmailPrimary(ctx, uid, newEmail.ID)
})
}

View File

@@ -46,22 +46,22 @@ func TestIsEmailUsed(t *testing.T) {
func TestMakeEmailPrimary(t *testing.T) {
assert.NoError(t, unittest.PrepareTestDatabase())
err := user_model.MakeActiveEmailPrimary(t.Context(), 9999999)
err := user_model.MakeActiveEmailPrimary(t.Context(), 1, 9999999)
assert.Error(t, err)
assert.ErrorIs(t, err, user_model.ErrEmailAddressNotExist{})
email := unittest.AssertExistsAndLoadBean(t, &user_model.EmailAddress{Email: "user11@example.com"})
err = user_model.MakeActiveEmailPrimary(t.Context(), email.ID)
err = user_model.MakeActiveEmailPrimary(t.Context(), email.UID, email.ID)
assert.Error(t, err)
assert.ErrorIs(t, err, user_model.ErrEmailAddressNotExist{}) // inactive email is considered as not exist for "MakeActiveEmailPrimary"
email = unittest.AssertExistsAndLoadBean(t, &user_model.EmailAddress{Email: "user9999999@example.com"})
err = user_model.MakeActiveEmailPrimary(t.Context(), email.ID)
err = user_model.MakeActiveEmailPrimary(t.Context(), email.UID, email.ID)
assert.Error(t, err)
assert.True(t, user_model.IsErrUserNotExist(err))
email = unittest.AssertExistsAndLoadBean(t, &user_model.EmailAddress{Email: "user101@example.com"})
err = user_model.MakeActiveEmailPrimary(t.Context(), email.ID)
err = user_model.MakeActiveEmailPrimary(t.Context(), email.UID, email.ID)
assert.NoError(t, err)
user, _ := user_model.GetUserByID(t.Context(), int64(10))

View File

@@ -102,7 +102,13 @@ func DeleteUserOpenID(ctx context.Context, openid *UserOpenID) (err error) {
}
// ToggleUserOpenIDVisibility toggles visibility of an openid address of given user.
func ToggleUserOpenIDVisibility(ctx context.Context, id int64) (err error) {
_, err = db.GetEngine(ctx).Exec("update `user_open_id` set `show` = not `show` where `id` = ?", id)
return err
func ToggleUserOpenIDVisibility(ctx context.Context, id int64, user *User) error {
affected, err := db.GetEngine(ctx).Exec("update `user_open_id` set `show` = not `show` where `id` = ? AND uid = ?", id, user.ID)
if err != nil {
return err
}
if n, _ := affected.RowsAffected(); n != 1 {
return util.NewNotExistErrorf("OpenID is unknown")
}
return nil
}

View File

@@ -8,6 +8,7 @@ import (
"code.gitea.io/gitea/models/unittest"
user_model "code.gitea.io/gitea/models/user"
"code.gitea.io/gitea/modules/util"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
@@ -33,12 +34,14 @@ func TestGetUserOpenIDs(t *testing.T) {
func TestToggleUserOpenIDVisibility(t *testing.T) {
assert.NoError(t, unittest.PrepareTestDatabase())
user, err := user_model.GetUserByID(t.Context(), int64(2))
require.NoError(t, err)
oids, err := user_model.GetUserOpenIDs(t.Context(), int64(2))
require.NoError(t, err)
require.Len(t, oids, 1)
assert.True(t, oids[0].Show)
err = user_model.ToggleUserOpenIDVisibility(t.Context(), oids[0].ID)
err = user_model.ToggleUserOpenIDVisibility(t.Context(), oids[0].ID, user)
require.NoError(t, err)
oids, err = user_model.GetUserOpenIDs(t.Context(), int64(2))
@@ -46,7 +49,7 @@ func TestToggleUserOpenIDVisibility(t *testing.T) {
require.Len(t, oids, 1)
assert.False(t, oids[0].Show)
err = user_model.ToggleUserOpenIDVisibility(t.Context(), oids[0].ID)
err = user_model.ToggleUserOpenIDVisibility(t.Context(), oids[0].ID, user)
require.NoError(t, err)
oids, err = user_model.GetUserOpenIDs(t.Context(), int64(2))
@@ -55,3 +58,13 @@ func TestToggleUserOpenIDVisibility(t *testing.T) {
assert.True(t, oids[0].Show)
}
}
func TestToggleUserOpenIDVisibilityRequiresOwnership(t *testing.T) {
assert.NoError(t, unittest.PrepareTestDatabase())
unauthorizedUser, err := user_model.GetUserByID(t.Context(), int64(2))
require.NoError(t, err)
err = user_model.ToggleUserOpenIDVisibility(t.Context(), int64(1), unauthorizedUser)
require.Error(t, err)
assert.ErrorIs(t, err, util.ErrNotExist)
}

View File

@@ -1444,3 +1444,15 @@ func DisabledFeaturesWithLoginType(user *User) *container.Set[string] {
}
return &setting.Admin.UserDisabledFeatures
}
// GetUserOrOrgByName returns the user or org by name
func GetUserOrOrgByName(ctx context.Context, name string) (*User, error) {
var u User
has, err := db.GetEngine(ctx).Where("lower_name = ?", strings.ToLower(name)).Get(&u)
if err != nil {
return nil, err
} else if !has {
return nil, ErrUserNotExist{Name: name}
}
return &u, nil
}

View File

@@ -26,9 +26,11 @@ type ConvertOpts struct {
KeepBOM bool
}
var ToUTF8WithFallbackReaderPrefetchSize = 16 * 1024
// ToUTF8WithFallbackReader detects the encoding of content and converts to UTF-8 reader if possible
func ToUTF8WithFallbackReader(rd io.Reader, opts ConvertOpts) io.Reader {
buf := make([]byte, 2048)
buf := make([]byte, ToUTF8WithFallbackReaderPrefetchSize)
n, err := util.ReadAtMost(rd, buf)
if err != nil {
return io.MultiReader(bytes.NewReader(MaybeRemoveBOM(buf[:n], opts)), rd)
@@ -41,6 +43,7 @@ func ToUTF8WithFallbackReader(rd io.Reader, opts ConvertOpts) io.Reader {
encoding, _ := charset.Lookup(charsetLabel)
if encoding == nil {
log.Error("Unknown encoding: %s", charsetLabel)
return io.MultiReader(bytes.NewReader(buf[:n]), rd)
}
@@ -54,17 +57,18 @@ func ToUTF8WithFallbackReader(rd io.Reader, opts ConvertOpts) io.Reader {
}
// ToUTF8 converts content to UTF8 encoding
func ToUTF8(content []byte, opts ConvertOpts) (string, error) {
func ToUTF8(content []byte, opts ConvertOpts) ([]byte, error) {
charsetLabel, err := DetectEncoding(content)
if err != nil {
return "", err
return content, err
} else if charsetLabel == "UTF-8" {
return string(MaybeRemoveBOM(content, opts)), nil
return MaybeRemoveBOM(content, opts), nil
}
encoding, _ := charset.Lookup(charsetLabel)
if encoding == nil {
return string(content), fmt.Errorf("Unknown encoding: %s", charsetLabel)
log.Error("Unknown encoding: %s", charsetLabel)
return content, fmt.Errorf("unknown encoding: %s", charsetLabel)
}
// If there is an error, we concatenate the nicely decoded part and the
@@ -76,7 +80,7 @@ func ToUTF8(content []byte, opts ConvertOpts) (string, error) {
result = MaybeRemoveBOM(result, opts)
return string(result), err
return result, err
}
// ToUTF8WithFallback detects the encoding of content and converts to UTF-8 if possible
@@ -94,6 +98,7 @@ func ToUTF8DropErrors(content []byte, opts ConvertOpts) []byte {
encoding, _ := charset.Lookup(charsetLabel)
if encoding == nil {
log.Error("Unknown encoding: %s", charsetLabel)
return content
}
@@ -130,28 +135,37 @@ func MaybeRemoveBOM(content []byte, opts ConvertOpts) []byte {
}
// DetectEncoding detect the encoding of content
func DetectEncoding(content []byte) (string, error) {
// it always returns a detected or guessed "encoding" string, no matter error happens or not
func DetectEncoding(content []byte) (encoding string, _ error) {
// First we check if the content represents valid utf8 content excepting a truncated character at the end.
// Now we could decode all the runes in turn but this is not necessarily the cheapest thing to do
// instead we walk backwards from the end to trim off a the incomplete character
// instead we walk backwards from the end to trim off the incomplete character
toValidate := content
end := len(toValidate) - 1
if end < 0 {
// no-op
} else if toValidate[end]>>5 == 0b110 {
// Incomplete 1 byte extension e.g. © <c2><a9> which has been truncated to <c2>
toValidate = toValidate[:end]
} else if end > 0 && toValidate[end]>>6 == 0b10 && toValidate[end-1]>>4 == 0b1110 {
// Incomplete 2 byte extension e.g. ⛔ <e2><9b><94> which has been truncated to <e2><9b>
toValidate = toValidate[:end-1]
} else if end > 1 && toValidate[end]>>6 == 0b10 && toValidate[end-1]>>6 == 0b10 && toValidate[end-2]>>3 == 0b11110 {
// Incomplete 3 byte extension e.g. 💩 <f0><9f><92><a9> which has been truncated to <f0><9f><92>
toValidate = toValidate[:end-2]
// U+0000 U+007F 0yyyzzzz
// U+0080 U+07FF 110xxxyy 10yyzzzz
// U+0800 U+FFFF 1110wwww 10xxxxyy 10yyzzzz
// U+010000 U+10FFFF 11110uvv 10vvwwww 10xxxxyy 10yyzzzz
cnt := 0
for end >= 0 && cnt < 4 {
c := toValidate[end]
if c>>5 == 0b110 || c>>4 == 0b1110 || c>>3 == 0b11110 {
// a leading byte
toValidate = toValidate[:end]
break
} else if c>>6 == 0b10 {
// a continuation byte
end--
} else {
// not an utf-8 byte
break
}
cnt++
}
if utf8.Valid(toValidate) {
log.Debug("Detected encoding: utf-8 (fast)")
return "UTF-8", nil
}
@@ -160,7 +174,7 @@ func DetectEncoding(content []byte) (string, error) {
if len(content) < 1024 {
// Check if original content is valid
if _, err := textDetector.DetectBest(content); err != nil {
return "", err
return util.IfZero(setting.Repository.AnsiCharset, "UTF-8"), err
}
times := 1024 / len(content)
detectContent = make([]byte, 0, times*len(content))
@@ -171,14 +185,10 @@ func DetectEncoding(content []byte) (string, error) {
detectContent = content
}
// Now we can't use DetectBest or just results[0] because the result isn't stable - so we need a tie break
// Now we can't use DetectBest or just results[0] because the result isn't stable - so we need a tie-break
results, err := textDetector.DetectAll(detectContent)
if err != nil {
if err == chardet.NotDetectedError && len(setting.Repository.AnsiCharset) > 0 {
log.Debug("Using default AnsiCharset: %s", setting.Repository.AnsiCharset)
return setting.Repository.AnsiCharset, nil
}
return "", err
return util.IfZero(setting.Repository.AnsiCharset, "UTF-8"), err
}
topConfidence := results[0].Confidence
@@ -201,11 +211,9 @@ func DetectEncoding(content []byte) (string, error) {
}
// FIXME: to properly decouple this function the fallback ANSI charset should be passed as an argument
if topResult.Charset != "UTF-8" && len(setting.Repository.AnsiCharset) > 0 {
log.Debug("Using default AnsiCharset: %s", setting.Repository.AnsiCharset)
if topResult.Charset != "UTF-8" && setting.Repository.AnsiCharset != "" {
return setting.Repository.AnsiCharset, err
}
log.Debug("Detected encoding: %s", topResult.Charset)
return topResult.Charset, err
return topResult.Charset, nil
}

View File

@@ -4,12 +4,12 @@
package charset
import (
"bytes"
"io"
"strings"
"testing"
"code.gitea.io/gitea/modules/setting"
"code.gitea.io/gitea/modules/test"
"github.com/stretchr/testify/assert"
)
@@ -47,12 +47,12 @@ func TestToUTF8(t *testing.T) {
res, err := ToUTF8([]byte{0x41, 0x42, 0x43}, ConvertOpts{})
assert.NoError(t, err)
assert.Equal(t, "ABC", res)
assert.Equal(t, "ABC", string(res))
// "áéíóú"
res, err = ToUTF8([]byte{0xc3, 0xa1, 0xc3, 0xa9, 0xc3, 0xad, 0xc3, 0xb3, 0xc3, 0xba}, ConvertOpts{})
assert.NoError(t, err)
assert.Equal(t, []byte{0xc3, 0xa1, 0xc3, 0xa9, 0xc3, 0xad, 0xc3, 0xb3, 0xc3, 0xba}, []byte(res))
assert.Equal(t, []byte{0xc3, 0xa1, 0xc3, 0xa9, 0xc3, 0xad, 0xc3, 0xb3, 0xc3, 0xba}, res)
// "áéíóú"
res, err = ToUTF8([]byte{
@@ -60,7 +60,7 @@ func TestToUTF8(t *testing.T) {
0xc3, 0xba,
}, ConvertOpts{})
assert.NoError(t, err)
assert.Equal(t, []byte{0xc3, 0xa1, 0xc3, 0xa9, 0xc3, 0xad, 0xc3, 0xb3, 0xc3, 0xba}, []byte(res))
assert.Equal(t, []byte{0xc3, 0xa1, 0xc3, 0xa9, 0xc3, 0xad, 0xc3, 0xb3, 0xc3, 0xba}, res)
res, err = ToUTF8([]byte{
0x48, 0x6F, 0x6C, 0x61, 0x2C, 0x20, 0x61, 0x73, 0xED, 0x20, 0x63,
@@ -96,12 +96,11 @@ func TestToUTF8(t *testing.T) {
assert.Equal(t, []byte{
0xE6, 0x97, 0xA5, 0xE5, 0xB1, 0x9E, 0xE7, 0xA7, 0x98, 0xE3,
0x81, 0x9E, 0xE3, 0x81, 0x97, 0xE3, 0x81, 0xA1, 0xE3, 0x82, 0x85, 0xE3, 0x80, 0x82,
},
[]byte(res))
}, res)
res, err = ToUTF8([]byte{0x00, 0x00, 0x00, 0x00}, ConvertOpts{})
assert.NoError(t, err)
assert.Equal(t, []byte{0x00, 0x00, 0x00, 0x00}, []byte(res))
assert.Equal(t, []byte{0x00, 0x00, 0x00, 0x00}, res)
}
func TestToUTF8WithFallback(t *testing.T) {
@@ -231,152 +230,42 @@ func TestDetectEncoding(t *testing.T) {
assert.Error(t, err)
}
func stringMustStartWith(t *testing.T, expected, value string) {
assert.Equal(t, expected, value[:len(expected)])
func stringMustStartWith(t *testing.T, expected string, value []byte) {
assert.Equal(t, expected, string(value[:len(expected)]))
}
func stringMustEndWith(t *testing.T, expected, value string) {
assert.Equal(t, expected, value[len(value)-len(expected):])
func stringMustEndWith(t *testing.T, expected string, value []byte) {
assert.Equal(t, expected, string(value[len(value)-len(expected):]))
}
func TestToUTF8WithFallbackReader(t *testing.T) {
resetDefaultCharsetsOrder()
test.MockVariableValue(&ToUTF8WithFallbackReaderPrefetchSize)
for testLen := range 2048 {
pattern := " test { () }\n"
input := ""
for len(input) < testLen {
input += pattern
}
input = input[:testLen]
input += "// Выключаем"
rd := ToUTF8WithFallbackReader(bytes.NewReader([]byte(input)), ConvertOpts{})
block := "aá啊🤔"
runes := []rune(block)
assert.Len(t, string(runes[0]), 1)
assert.Len(t, string(runes[1]), 2)
assert.Len(t, string(runes[2]), 3)
assert.Len(t, string(runes[3]), 4)
content := strings.Repeat(block, 2)
for i := 1; i < len(content); i++ {
encoding, err := DetectEncoding([]byte(content[:i]))
assert.NoError(t, err)
assert.Equal(t, "UTF-8", encoding)
ToUTF8WithFallbackReaderPrefetchSize = i
rd := ToUTF8WithFallbackReader(strings.NewReader(content), ConvertOpts{})
r, _ := io.ReadAll(rd)
assert.Equalf(t, input, string(r), "testing string len=%d", testLen)
assert.Equal(t, content, string(r))
}
for _, r := range runes {
content = "abc abc " + string(r) + string(r) + string(r)
for i := 0; i < len(content); i++ {
encoding, err := DetectEncoding([]byte(content[:i]))
assert.NoError(t, err)
assert.Equal(t, "UTF-8", encoding)
}
}
truncatedOneByteExtension := failFastBytes
encoding, _ := DetectEncoding(truncatedOneByteExtension)
assert.Equal(t, "UTF-8", encoding)
truncatedTwoByteExtension := failFastBytes
truncatedTwoByteExtension[len(failFastBytes)-1] = 0x9b
truncatedTwoByteExtension[len(failFastBytes)-2] = 0xe2
encoding, _ = DetectEncoding(truncatedTwoByteExtension)
assert.Equal(t, "UTF-8", encoding)
truncatedThreeByteExtension := failFastBytes
truncatedThreeByteExtension[len(failFastBytes)-1] = 0x92
truncatedThreeByteExtension[len(failFastBytes)-2] = 0x9f
truncatedThreeByteExtension[len(failFastBytes)-3] = 0xf0
encoding, _ = DetectEncoding(truncatedThreeByteExtension)
assert.Equal(t, "UTF-8", encoding)
}
var failFastBytes = []byte{
0x69, 0x6d, 0x70, 0x6f, 0x72, 0x74, 0x20, 0x6f, 0x72, 0x67, 0x2e, 0x61, 0x70, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x74, 0x6f,
0x6f, 0x6c, 0x73, 0x2e, 0x61, 0x6e, 0x74, 0x2e, 0x74, 0x61, 0x73, 0x6b, 0x64, 0x65, 0x66, 0x73, 0x2e, 0x63, 0x6f, 0x6e,
0x64, 0x69, 0x74, 0x69, 0x6f, 0x6e, 0x2e, 0x4f, 0x73, 0x0a, 0x69, 0x6d, 0x70, 0x6f, 0x72, 0x74, 0x20, 0x6f, 0x72, 0x67,
0x2e, 0x73, 0x70, 0x72, 0x69, 0x6e, 0x67, 0x66, 0x72, 0x61, 0x6d, 0x65, 0x77, 0x6f, 0x72, 0x6b, 0x2e, 0x62, 0x6f, 0x6f,
0x74, 0x2e, 0x67, 0x72, 0x61, 0x64, 0x6c, 0x65, 0x2e, 0x74, 0x61, 0x73, 0x6b, 0x73, 0x2e, 0x72, 0x75, 0x6e, 0x2e, 0x42,
0x6f, 0x6f, 0x74, 0x52, 0x75, 0x6e, 0x0a, 0x0a, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x73, 0x20, 0x7b, 0x0a, 0x20, 0x20,
0x20, 0x20, 0x69, 0x64, 0x28, 0x22, 0x6f, 0x72, 0x67, 0x2e, 0x73, 0x70, 0x72, 0x69, 0x6e, 0x67, 0x66, 0x72, 0x61, 0x6d,
0x65, 0x77, 0x6f, 0x72, 0x6b, 0x2e, 0x62, 0x6f, 0x6f, 0x74, 0x22, 0x29, 0x0a, 0x7d, 0x0a, 0x0a, 0x64, 0x65, 0x70, 0x65,
0x6e, 0x64, 0x65, 0x6e, 0x63, 0x69, 0x65, 0x73, 0x20, 0x7b, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x69, 0x6d, 0x70, 0x6c, 0x65,
0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x28, 0x70, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x28, 0x22, 0x3a,
0x73, 0x65, 0x72, 0x76, 0x65, 0x72, 0x3a, 0x61, 0x70, 0x69, 0x22, 0x29, 0x29, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x69, 0x6d,
0x70, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x28, 0x70, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74,
0x28, 0x22, 0x3a, 0x73, 0x65, 0x72, 0x76, 0x65, 0x72, 0x3a, 0x61, 0x70, 0x69, 0x2d, 0x64, 0x6f, 0x63, 0x73, 0x22, 0x29,
0x29, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x69, 0x6d, 0x70, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e,
0x28, 0x70, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x28, 0x22, 0x3a, 0x73, 0x65, 0x72, 0x76, 0x65, 0x72, 0x3a, 0x64, 0x62,
0x22, 0x29, 0x29, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x69, 0x6d, 0x70, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69,
0x6f, 0x6e, 0x28, 0x70, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x28, 0x22, 0x3a, 0x73, 0x65, 0x72, 0x76, 0x65, 0x72, 0x3a,
0x73, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x22, 0x29, 0x29, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x69, 0x6d, 0x70, 0x6c, 0x65,
0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x28, 0x70, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x28, 0x22, 0x3a,
0x73, 0x65, 0x72, 0x76, 0x65, 0x72, 0x3a, 0x69, 0x6e, 0x74, 0x65, 0x67, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x2d, 0x66,
0x73, 0x22, 0x29, 0x29, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x69, 0x6d, 0x70, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74,
0x69, 0x6f, 0x6e, 0x28, 0x70, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x28, 0x22, 0x3a, 0x73, 0x65, 0x72, 0x76, 0x65, 0x72,
0x3a, 0x69, 0x6e, 0x74, 0x65, 0x67, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x2d, 0x6d, 0x71, 0x22, 0x29, 0x29, 0x0a, 0x0a,
0x20, 0x20, 0x20, 0x20, 0x69, 0x6d, 0x70, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x28, 0x22,
0x6a, 0x66, 0x75, 0x73, 0x69, 0x6f, 0x6e, 0x2e, 0x70, 0x65, 0x3a, 0x70, 0x65, 0x2d, 0x63, 0x6f, 0x6d, 0x6d, 0x6f, 0x6e,
0x2d, 0x61, 0x75, 0x74, 0x68, 0x2d, 0x72, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x2d, 0x73, 0x74, 0x61, 0x72, 0x74,
0x65, 0x72, 0x22, 0x29, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x69, 0x6d, 0x70, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74,
0x69, 0x6f, 0x6e, 0x28, 0x22, 0x6a, 0x66, 0x75, 0x73, 0x69, 0x6f, 0x6e, 0x2e, 0x70, 0x65, 0x3a, 0x70, 0x65, 0x2d, 0x63,
0x6f, 0x6d, 0x6d, 0x6f, 0x6e, 0x2d, 0x68, 0x61, 0x6c, 0x22, 0x29, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x69, 0x6d, 0x70, 0x6c,
0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x28, 0x22, 0x6a, 0x66, 0x75, 0x73, 0x69, 0x6f, 0x6e, 0x2e,
0x70, 0x65, 0x3a, 0x70, 0x65, 0x2d, 0x63, 0x6f, 0x6d, 0x6d, 0x6f, 0x6e, 0x2d, 0x63, 0x6f, 0x72, 0x65, 0x22, 0x29, 0x0a,
0x0a, 0x20, 0x20, 0x20, 0x20, 0x69, 0x6d, 0x70, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x28,
0x22, 0x6f, 0x72, 0x67, 0x2e, 0x73, 0x70, 0x72, 0x69, 0x6e, 0x67, 0x66, 0x72, 0x61, 0x6d, 0x65, 0x77, 0x6f, 0x72, 0x6b,
0x2e, 0x62, 0x6f, 0x6f, 0x74, 0x3a, 0x73, 0x70, 0x72, 0x69, 0x6e, 0x67, 0x2d, 0x62, 0x6f, 0x6f, 0x74, 0x2d, 0x73, 0x74,
0x61, 0x72, 0x74, 0x65, 0x72, 0x2d, 0x77, 0x65, 0x62, 0x22, 0x29, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x69, 0x6d, 0x70, 0x6c,
0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x28, 0x22, 0x6f, 0x72, 0x67, 0x2e, 0x73, 0x70, 0x72, 0x69,
0x6e, 0x67, 0x66, 0x72, 0x61, 0x6d, 0x65, 0x77, 0x6f, 0x72, 0x6b, 0x2e, 0x62, 0x6f, 0x6f, 0x74, 0x3a, 0x73, 0x70, 0x72,
0x69, 0x6e, 0x67, 0x2d, 0x62, 0x6f, 0x6f, 0x74, 0x2d, 0x73, 0x74, 0x61, 0x72, 0x74, 0x65, 0x72, 0x2d, 0x61, 0x6f, 0x70,
0x22, 0x29, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x69, 0x6d, 0x70, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f,
0x6e, 0x28, 0x22, 0x6f, 0x72, 0x67, 0x2e, 0x73, 0x70, 0x72, 0x69, 0x6e, 0x67, 0x66, 0x72, 0x61, 0x6d, 0x65, 0x77, 0x6f,
0x72, 0x6b, 0x2e, 0x62, 0x6f, 0x6f, 0x74, 0x3a, 0x73, 0x70, 0x72, 0x69, 0x6e, 0x67, 0x2d, 0x62, 0x6f, 0x6f, 0x74, 0x2d,
0x73, 0x74, 0x61, 0x72, 0x74, 0x65, 0x72, 0x2d, 0x61, 0x63, 0x74, 0x75, 0x61, 0x74, 0x6f, 0x72, 0x22, 0x29, 0x0a, 0x20,
0x20, 0x20, 0x20, 0x69, 0x6d, 0x70, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x28, 0x22, 0x6f,
0x72, 0x67, 0x2e, 0x73, 0x70, 0x72, 0x69, 0x6e, 0x67, 0x66, 0x72, 0x61, 0x6d, 0x65, 0x77, 0x6f, 0x72, 0x6b, 0x2e, 0x63,
0x6c, 0x6f, 0x75, 0x64, 0x3a, 0x73, 0x70, 0x72, 0x69, 0x6e, 0x67, 0x2d, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x2d, 0x73, 0x74,
0x61, 0x72, 0x74, 0x65, 0x72, 0x2d, 0x62, 0x6f, 0x6f, 0x74, 0x73, 0x74, 0x72, 0x61, 0x70, 0x22, 0x29, 0x0a, 0x20, 0x20,
0x20, 0x20, 0x69, 0x6d, 0x70, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x28, 0x22, 0x6f, 0x72,
0x67, 0x2e, 0x73, 0x70, 0x72, 0x69, 0x6e, 0x67, 0x66, 0x72, 0x61, 0x6d, 0x65, 0x77, 0x6f, 0x72, 0x6b, 0x2e, 0x63, 0x6c,
0x6f, 0x75, 0x64, 0x3a, 0x73, 0x70, 0x72, 0x69, 0x6e, 0x67, 0x2d, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x2d, 0x73, 0x74, 0x61,
0x72, 0x74, 0x65, 0x72, 0x2d, 0x63, 0x6f, 0x6e, 0x73, 0x75, 0x6c, 0x2d, 0x61, 0x6c, 0x6c, 0x22, 0x29, 0x0a, 0x20, 0x20,
0x20, 0x20, 0x69, 0x6d, 0x70, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x28, 0x22, 0x6f, 0x72,
0x67, 0x2e, 0x73, 0x70, 0x72, 0x69, 0x6e, 0x67, 0x66, 0x72, 0x61, 0x6d, 0x65, 0x77, 0x6f, 0x72, 0x6b, 0x2e, 0x63, 0x6c,
0x6f, 0x75, 0x64, 0x3a, 0x73, 0x70, 0x72, 0x69, 0x6e, 0x67, 0x2d, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x2d, 0x73, 0x74, 0x61,
0x72, 0x74, 0x65, 0x72, 0x2d, 0x73, 0x6c, 0x65, 0x75, 0x74, 0x68, 0x22, 0x29, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x69, 0x6d,
0x70, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x28, 0x22, 0x6f, 0x72, 0x67, 0x2e, 0x73, 0x70,
0x72, 0x69, 0x6e, 0x67, 0x66, 0x72, 0x61, 0x6d, 0x65, 0x77, 0x6f, 0x72, 0x6b, 0x2e, 0x72, 0x65, 0x74, 0x72, 0x79, 0x3a,
0x73, 0x70, 0x72, 0x69, 0x6e, 0x67, 0x2d, 0x72, 0x65, 0x74, 0x72, 0x79, 0x22, 0x29, 0x0a, 0x0a, 0x20, 0x20, 0x20, 0x20,
0x69, 0x6d, 0x70, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x28, 0x22, 0x63, 0x68, 0x2e, 0x71,
0x6f, 0x73, 0x2e, 0x6c, 0x6f, 0x67, 0x62, 0x61, 0x63, 0x6b, 0x3a, 0x6c, 0x6f, 0x67, 0x62, 0x61, 0x63, 0x6b, 0x2d, 0x63,
0x6c, 0x61, 0x73, 0x73, 0x69, 0x63, 0x22, 0x29, 0x0a, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x69, 0x6d, 0x70, 0x6c, 0x65, 0x6d,
0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x28, 0x22, 0x69, 0x6f, 0x2e, 0x6d, 0x69, 0x63, 0x72, 0x6f, 0x6d, 0x65,
0x74, 0x65, 0x72, 0x3a, 0x6d, 0x69, 0x63, 0x72, 0x6f, 0x6d, 0x65, 0x74, 0x65, 0x72, 0x2d, 0x72, 0x65, 0x67, 0x69, 0x73,
0x74, 0x72, 0x79, 0x2d, 0x70, 0x72, 0x6f, 0x6d, 0x65, 0x74, 0x68, 0x65, 0x75, 0x73, 0x22, 0x29, 0x0a, 0x0a, 0x20, 0x20,
0x20, 0x20, 0x69, 0x6d, 0x70, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x28, 0x6b, 0x6f, 0x74,
0x6c, 0x69, 0x6e, 0x28, 0x22, 0x73, 0x74, 0x64, 0x6c, 0x69, 0x62, 0x22, 0x29, 0x29, 0x0a, 0x0a, 0x20, 0x20, 0x20, 0x20,
0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f,
0x2f, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x2f, 0x2f, 0x20, 0x54, 0x65, 0x73, 0x74, 0x20, 0x64, 0x65, 0x70, 0x65, 0x6e, 0x64,
0x65, 0x6e, 0x63, 0x69, 0x65, 0x73, 0x2e, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f,
0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x0a, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x74,
0x65, 0x73, 0x74, 0x49, 0x6d, 0x70, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x28, 0x22, 0x6a,
0x66, 0x75, 0x73, 0x69, 0x6f, 0x6e, 0x2e, 0x70, 0x65, 0x3a, 0x70, 0x65, 0x2d, 0x63, 0x6f, 0x6d, 0x6d, 0x6f, 0x6e, 0x2d,
0x74, 0x65, 0x73, 0x74, 0x22, 0x29, 0x0a, 0x7d, 0x0a, 0x0a, 0x76, 0x61, 0x6c, 0x20, 0x70, 0x61, 0x74, 0x63, 0x68, 0x4a,
0x61, 0x72, 0x20, 0x62, 0x79, 0x20, 0x74, 0x61, 0x73, 0x6b, 0x73, 0x2e, 0x72, 0x65, 0x67, 0x69, 0x73, 0x74, 0x65, 0x72,
0x69, 0x6e, 0x67, 0x28, 0x4a, 0x61, 0x72, 0x3a, 0x3a, 0x63, 0x6c, 0x61, 0x73, 0x73, 0x29, 0x20, 0x7b, 0x0a, 0x20, 0x20,
0x20, 0x20, 0x61, 0x72, 0x63, 0x68, 0x69, 0x76, 0x65, 0x43, 0x6c, 0x61, 0x73, 0x73, 0x69, 0x66, 0x69, 0x65, 0x72, 0x2e,
0x73, 0x65, 0x74, 0x28, 0x22, 0x70, 0x61, 0x74, 0x63, 0x68, 0x65, 0x64, 0x22, 0x29, 0x0a, 0x0a, 0x20, 0x20, 0x20, 0x20,
0x76, 0x61, 0x6c, 0x20, 0x72, 0x75, 0x6e, 0x74, 0x69, 0x6d, 0x65, 0x43, 0x6c, 0x61, 0x73, 0x73, 0x70, 0x61, 0x74, 0x68,
0x20, 0x62, 0x79, 0x20, 0x63, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x75, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x73, 0x2e, 0x67,
0x65, 0x74, 0x74, 0x69, 0x6e, 0x67, 0x0a, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x6d, 0x61, 0x6e, 0x69, 0x66, 0x65, 0x73, 0x74,
0x20, 0x7b, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x61, 0x74, 0x74, 0x72, 0x69, 0x62, 0x75, 0x74, 0x65,
0x73, 0x28, 0x22, 0x43, 0x6c, 0x61, 0x73, 0x73, 0x2d, 0x50, 0x61, 0x74, 0x68, 0x22, 0x20, 0x74, 0x6f, 0x20, 0x6f, 0x62,
0x6a, 0x65, 0x63, 0x74, 0x20, 0x7b, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x70,
0x72, 0x69, 0x76, 0x61, 0x74, 0x65, 0x20, 0x76, 0x61, 0x6c, 0x20, 0x70, 0x61, 0x74, 0x74, 0x65, 0x72, 0x6e, 0x20, 0x3d,
0x20, 0x22, 0x66, 0x69, 0x6c, 0x65, 0x3a, 0x2f, 0x2b, 0x22, 0x2e, 0x74, 0x6f, 0x52, 0x65, 0x67, 0x65, 0x78, 0x28, 0x29,
0x0a, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x6f, 0x76, 0x65, 0x72, 0x72, 0x69, 0x64,
0x65, 0x20, 0x66, 0x75, 0x6e, 0x20, 0x74, 0x6f, 0x53, 0x74, 0x72, 0x69, 0x6e, 0x67, 0x28, 0x29, 0x3a, 0x20, 0x53, 0x74,
0x72, 0x69, 0x6e, 0x67, 0x20, 0x3d, 0x20, 0x72, 0x75, 0x6e, 0x74, 0x69, 0x6d, 0x65, 0x43, 0x6c, 0x61, 0x73, 0x73, 0x70,
0x61, 0x74, 0x68, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x73, 0x2e, 0x6a, 0x6f, 0x69, 0x6e, 0x54, 0x6f, 0x53, 0x74, 0x72, 0x69,
0x6e, 0x67, 0x28, 0x22, 0x20, 0x22, 0x29, 0x20, 0x7b, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20,
0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x69, 0x74, 0x2e, 0x74, 0x6f, 0x55, 0x52, 0x49, 0x28, 0x29, 0x2e, 0x74, 0x6f, 0x55,
0x52, 0x4c, 0x28, 0x29, 0x2e, 0x74, 0x6f, 0x53, 0x74, 0x72, 0x69, 0x6e, 0x67, 0x28, 0x29, 0x2e, 0x72, 0x65, 0x70, 0x6c,
0x61, 0x63, 0x65, 0x46, 0x69, 0x72, 0x73, 0x74, 0x28, 0x70, 0x61, 0x74, 0x74, 0x65, 0x72, 0x6e, 0x2c, 0x20, 0x22, 0x2f,
0x22, 0x29, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x7d, 0x0a, 0x20, 0x20, 0x20,
0x20, 0x20, 0x20, 0x20, 0x20, 0x7d, 0x29, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x7d, 0x0a, 0x7d, 0x0a, 0x0a, 0x74, 0x61, 0x73,
0x6b, 0x73, 0x2e, 0x6e, 0x61, 0x6d, 0x65, 0x64, 0x3c, 0x42, 0x6f, 0x6f, 0x74, 0x52, 0x75, 0x6e, 0x3e, 0x28, 0x22, 0x62,
0x6f, 0x6f, 0x74, 0x52, 0x75, 0x6e, 0x22, 0x29, 0x20, 0x7b, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x69, 0x66, 0x20, 0x28, 0x4f,
0x73, 0x2e, 0x69, 0x73, 0x46, 0x61, 0x6d, 0x69, 0x6c, 0x79, 0x28, 0x4f, 0x73, 0x2e, 0x46, 0x41, 0x4d, 0x49, 0x4c, 0x59,
0x5f, 0x57, 0x49, 0x4e, 0x44, 0x4f, 0x57, 0x53, 0x29, 0x29, 0x20, 0x7b, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20,
0x20, 0x63, 0x6c, 0x61, 0x73, 0x73, 0x70, 0x61, 0x74, 0x68, 0x20, 0x3d, 0x20, 0x66, 0x69, 0x6c, 0x65, 0x73, 0x28, 0x73,
0x6f, 0x75, 0x72, 0x63, 0x65, 0x53, 0x65, 0x74, 0x73, 0x2e, 0x6e, 0x61, 0x6d, 0x65, 0x64, 0x28, 0x22, 0x6d, 0x61, 0x69,
0x6e, 0x22, 0x29, 0x2e, 0x6d, 0x61, 0x70, 0x20, 0x7b, 0x20, 0x69, 0x74, 0x2e, 0x6f, 0x75, 0x74, 0x70, 0x75, 0x74, 0x20,
0x7d, 0x2c, 0x20, 0x70, 0x61, 0x74, 0x63, 0x68, 0x4a, 0x61, 0x72, 0x29, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x7d, 0x0a, 0x0a,
0x20, 0x20, 0x20, 0x20, 0x2f, 0x2f, 0x20, 0xd0,
}

View File

@@ -9,6 +9,7 @@ import (
activities_model "code.gitea.io/gitea/models/activities"
issues_model "code.gitea.io/gitea/models/issues"
user_model "code.gitea.io/gitea/models/user"
"code.gitea.io/gitea/modules/graceful"
"code.gitea.io/gitea/modules/json"
"code.gitea.io/gitea/modules/log"
@@ -91,7 +92,13 @@ loop:
}
for _, userStopwatches := range usersStopwatches {
apiSWs, err := convert.ToStopWatches(ctx, userStopwatches.StopWatches)
u, err := user_model.GetUserByID(ctx, userStopwatches.UserID)
if err != nil {
log.Error("Unable to get user %d: %v", userStopwatches.UserID, err)
continue
}
apiSWs, err := convert.ToStopWatches(ctx, u, userStopwatches.StopWatches)
if err != nil {
if !issues_model.IsErrIssueNotExist(err) {
log.Error("Unable to APIFormat stopwatches: %v", err)

View File

@@ -76,7 +76,7 @@ func (m *MaterialIconProvider) renderFileIconSVG(p *RenderedIconPool, name, svg,
if p.IconSVGs[svgID] == "" {
p.IconSVGs[svgID] = svgHTML
}
return template.HTML(`<svg ` + svgCommonAttrs + `><use xlink:href="#` + svgID + `"></use></svg>`)
return template.HTML(`<svg ` + svgCommonAttrs + `><use href="#` + svgID + `"></use></svg>`)
}
func (m *MaterialIconProvider) EntryIconHTML(p *RenderedIconPool, entry *EntryInfo) template.HTML {

View File

@@ -25,7 +25,7 @@ func (p *RenderedIconPool) RenderToHTML() template.HTML {
return ""
}
sb := &strings.Builder{}
sb.WriteString(`<div class=tw-hidden>`)
sb.WriteString(`<div class="svg-icon-container">`)
for _, icon := range p.IconSVGs {
sb.WriteString(string(icon))
}

View File

@@ -9,24 +9,38 @@ package git
import (
"io"
"code.gitea.io/gitea/modules/log"
"github.com/go-git/go-git/v5/plumbing"
)
// Blob represents a Git object.
type Blob struct {
ID ObjectID
ID ObjectID
repo *Repository
name string
}
gogitEncodedObj plumbing.EncodedObject
name string
func (b *Blob) gogitEncodedObj() (plumbing.EncodedObject, error) {
return b.repo.gogitRepo.Storer.EncodedObject(plumbing.AnyObject, plumbing.Hash(b.ID.RawValue()))
}
// DataAsync gets a ReadCloser for the contents of a blob without reading it all.
// Calling the Close function on the result will discard all unread output.
func (b *Blob) DataAsync() (io.ReadCloser, error) {
return b.gogitEncodedObj.Reader()
obj, err := b.gogitEncodedObj()
if err != nil {
return nil, err
}
return obj.Reader()
}
// Size returns the uncompressed size of the blob
func (b *Blob) Size() int64 {
return b.gogitEncodedObj.Size()
obj, err := b.gogitEncodedObj()
if err != nil {
log.Error("Error getting gogit encoded object for blob %s(%s): %v", b.name, b.ID.String(), err)
return 0
}
return obj.Size()
}

View File

@@ -30,9 +30,11 @@ type Parser struct {
func NewParser(r io.Reader, format Format) *Parser {
scanner := bufio.NewScanner(r)
// default MaxScanTokenSize = 64 kiB may be too small for some references,
// so allow the buffer to grow up to 4x if needed
scanner.Buffer(nil, 4*bufio.MaxScanTokenSize)
// default Scanner.MaxScanTokenSize = 64 kiB may be too small for some references,
// so allow the buffer to be large enough in case the ref has long content (e.g.: a tag with long message)
// as long as it doesn't exceed some reasonable limit (4 MiB here, or MAX_DISPLAY_FILE_SIZE=8MiB), it is OK
// there are still some choices: 1. add a config option for the limit; 2. don't use scanner and write our own parser to fully handle large contents
scanner.Buffer(nil, 4*1024*1024)
// in addition to the reference delimiter we specified in the --format,
// `git for-each-ref` will always add a newline after every reference.

View File

@@ -13,6 +13,7 @@ import (
"slices"
"strconv"
"strings"
"time"
"code.gitea.io/gitea/modules/git/gitcmd"
"code.gitea.io/gitea/modules/util"
@@ -41,6 +42,10 @@ type GrepOptions struct {
PathspecList []string
}
// grepSearchTimeout is the timeout for git grep search, it should be long enough to get results
// but not too long to cause performance issues
const grepSearchTimeout = 30 * time.Second
func GrepSearch(ctx context.Context, repo *Repository, search string, opts GrepOptions) ([]*GrepResult, error) {
stdoutReader, stdoutWriter, err := os.Pipe()
if err != nil {
@@ -85,9 +90,10 @@ func GrepSearch(ctx context.Context, repo *Repository, search string, opts GrepO
opts.MaxResultLimit = util.IfZero(opts.MaxResultLimit, 50)
stderr := bytes.Buffer{}
err = cmd.Run(ctx, &gitcmd.RunOpts{
Dir: repo.Path,
Stdout: stdoutWriter,
Stderr: &stderr,
Dir: repo.Path,
Stdout: stdoutWriter,
Stderr: &stderr,
Timeout: grepSearchTimeout,
PipelineFunc: func(ctx context.Context, cancel context.CancelFunc) error {
_ = stdoutWriter.Close()
defer stdoutReader.Close()

View File

@@ -9,5 +9,11 @@ func (repo *Repository) GetBlob(idStr string) (*Blob, error) {
if err != nil {
return nil, err
}
return repo.getBlob(id)
if id.IsZero() {
return nil, ErrNotExist{id.String(), ""}
}
return &Blob{
ID: id,
repo: repo,
}, nil
}

View File

@@ -1,22 +0,0 @@
// Copyright 2018 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
//go:build gogit
package git
import (
"github.com/go-git/go-git/v5/plumbing"
)
func (repo *Repository) getBlob(id ObjectID) (*Blob, error) {
encodedObj, err := repo.gogitRepo.Storer.EncodedObject(plumbing.AnyObject, plumbing.Hash(id.RawValue()))
if err != nil {
return nil, ErrNotExist{id.String(), ""}
}
return &Blob{
ID: id,
gogitEncodedObj: encodedObj,
}, nil
}

View File

@@ -1,16 +0,0 @@
// Copyright 2020 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
//go:build !gogit
package git
func (repo *Repository) getBlob(id ObjectID) (*Blob, error) {
if id.IsZero() {
return nil, ErrNotExist{id.String(), ""}
}
return &Blob{
ID: id,
repo: repo,
}, nil
}

View File

@@ -7,7 +7,6 @@
package git
import (
"github.com/go-git/go-git/v5/plumbing"
"github.com/go-git/go-git/v5/plumbing/filemode"
"github.com/go-git/go-git/v5/plumbing/object"
)
@@ -83,14 +82,9 @@ func (te *TreeEntry) IsExecutable() bool {
// Blob returns the blob object the entry
func (te *TreeEntry) Blob() *Blob {
encodedObj, err := te.ptree.repo.gogitRepo.Storer.EncodedObject(plumbing.AnyObject, te.gogitTreeEntry.Hash)
if err != nil {
return nil
}
return &Blob{
ID: ParseGogitHash(te.gogitTreeEntry.Hash),
gogitEncodedObj: encodedObj,
name: te.Name(),
ID: ParseGogitHash(te.gogitTreeEntry.Hash),
repo: te.ptree.repo,
name: te.Name(),
}
}

View File

@@ -24,7 +24,18 @@ func urlIsRelative(s string, u *url.URL) bool {
if len(s) > 1 && (s[0] == '/' || s[0] == '\\') && (s[1] == '/' || s[1] == '\\') {
return false
}
return u != nil && u.Scheme == "" && u.Host == ""
if u == nil {
return false // invalid URL
}
if u.Scheme != "" || u.Host != "" {
return false // absolute URL with scheme or host
}
// Now, the URL is likely a relative URL
// HINT: GOLANG-HTTP-REDIRECT-BUG: Golang security vulnerability: "http.Redirect" calls "path.Clean" and changes the meaning of a path
// For example, `/a/../\b` will be changed to `/\b`, then it hits the first checked pattern and becomes an open redirect to "{current-scheme}://b"
// For a valid relative URL, its "path" shouldn't contain `\` because such char must be escaped.
// So if the "path" contains `\`, it is not a valid relative URL, then we can prevent open redirect.
return !strings.Contains(u.Path, "\\")
}
// IsRelativeURL detects if a URL is relative (no scheme or host)
@@ -35,14 +46,14 @@ func IsRelativeURL(s string) bool {
func getRequestScheme(req *http.Request) string {
// https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-Proto
if s := req.Header.Get("X-Forwarded-Proto"); s != "" {
return s
if proto, ok := parseForwardedProtoValue(req.Header.Get("X-Forwarded-Proto")); ok {
return proto
}
if s := req.Header.Get("X-Forwarded-Protocol"); s != "" {
return s
if proto, ok := parseForwardedProtoValue(req.Header.Get("X-Forwarded-Protocol")); ok {
return proto
}
if s := req.Header.Get("X-Url-Scheme"); s != "" {
return s
if proto, ok := parseForwardedProtoValue(req.Header.Get("X-Url-Scheme")); ok {
return proto
}
if s := req.Header.Get("Front-End-Https"); s != "" {
return util.Iif(s == "on", "https", "http")
@@ -53,6 +64,13 @@ func getRequestScheme(req *http.Request) string {
return ""
}
func parseForwardedProtoValue(val string) (string, bool) {
if val == "http" || val == "https" {
return val, true
}
return "", false
}
// GuessCurrentAppURL tries to guess the current full public URL (with sub-path) by http headers. It always has a '/' suffix, exactly the same as setting.AppURL
// TODO: should rename it to GuessCurrentPublicURL in the future
func GuessCurrentAppURL(ctx context.Context) string {

View File

@@ -23,6 +23,7 @@ func TestIsRelativeURL(t *testing.T) {
"foo",
"/",
"/foo?k=%20#abc",
"/foo?k=\\",
}
for _, s := range rel {
assert.True(t, IsRelativeURL(s), "rel = %q", s)
@@ -32,6 +33,8 @@ func TestIsRelativeURL(t *testing.T) {
"\\\\",
"/\\",
"\\/",
"/a/../\\b",
"/any\\thing",
"mailto:a@b.com",
"https://test.com",
}
@@ -44,6 +47,7 @@ func TestGuessCurrentHostURL(t *testing.T) {
defer test.MockVariableValue(&setting.AppURL, "http://cfg-host/sub/")()
defer test.MockVariableValue(&setting.AppSubURL, "/sub")()
headersWithProto := http.Header{"X-Forwarded-Proto": {"https"}}
maliciousProtoHeaders := http.Header{"X-Forwarded-Proto": {"http://attacker.host/?trash="}}
t.Run("Legacy", func(t *testing.T) {
defer test.MockVariableValue(&setting.PublicURLDetection, setting.PublicURLLegacy)()
@@ -57,6 +61,9 @@ func TestGuessCurrentHostURL(t *testing.T) {
// if "X-Forwarded-Proto" exists, then use it and "Host" header
ctx = context.WithValue(t.Context(), RequestContextKey, &http.Request{Host: "req-host:3000", Header: headersWithProto})
assert.Equal(t, "https://req-host:3000", GuessCurrentHostURL(ctx))
ctx = context.WithValue(t.Context(), RequestContextKey, &http.Request{Host: "req-host:3000", Header: maliciousProtoHeaders})
assert.Equal(t, "http://cfg-host", GuessCurrentHostURL(ctx))
})
t.Run("Auto", func(t *testing.T) {
@@ -73,6 +80,9 @@ func TestGuessCurrentHostURL(t *testing.T) {
ctx = context.WithValue(t.Context(), RequestContextKey, &http.Request{Host: "req-host:3000", Header: headersWithProto})
assert.Equal(t, "https://req-host:3000", GuessCurrentHostURL(ctx))
ctx = context.WithValue(t.Context(), RequestContextKey, &http.Request{Host: "req-host:3000", Header: maliciousProtoHeaders})
assert.Equal(t, "http://req-host:3000", GuessCurrentHostURL(ctx))
})
}

View File

@@ -30,6 +30,10 @@ func TestMathRender(t *testing.T) {
"$ a $",
`<p><code class="language-math">a</code></p>` + nl,
},
{
"$a$$b$",
`<p><code class="language-math">a</code><code class="language-math">b</code></p>` + nl,
},
{
"$a$ $b$",
`<p><code class="language-math">a</code> <code class="language-math">b</code></p>` + nl,
@@ -59,7 +63,7 @@ func TestMathRender(t *testing.T) {
`<p>a$b $a a$b b$</p>` + nl,
},
{
"a$x$",
"a$x$", // Pattern: "word$other$" The real world example is: "Price is between US$1 and US$2.", so don't parse this.
`<p>a$x$</p>` + nl,
},
{
@@ -70,6 +74,10 @@ func TestMathRender(t *testing.T) {
"$a$ ($b$) [$c$] {$d$}",
`<p><code class="language-math">a</code> (<code class="language-math">b</code>) [$c$] {$d$}</p>` + nl,
},
{
"[$a$](link)",
`<p><a href="/link" rel="nofollow"><code class="language-math">a</code></a></p>` + nl,
},
{
"$$a$$",
`<p><code class="language-math">a</code></p>` + nl,

View File

@@ -54,6 +54,10 @@ func isAlphanumeric(b byte) bool {
return (b >= 'a' && b <= 'z') || (b >= 'A' && b <= 'Z') || (b >= '0' && b <= '9')
}
func isInMarkdownLinkText(block text.Reader, lineAfter []byte) bool {
return block.PrecendingCharacter() == '[' && bytes.HasPrefix(lineAfter, []byte("]("))
}
// Parse parses the current line and returns a result of parsing.
func (parser *inlineParser) Parse(parent ast.Node, block text.Reader, pc parser.Context) ast.Node {
line, _ := block.PeekLine()
@@ -115,7 +119,9 @@ func (parser *inlineParser) Parse(parent ast.Node, block text.Reader, pc parser.
}
// check valid ending character
isValidEndingChar := isPunctuation(succeedingCharacter) || isParenthesesClose(succeedingCharacter) ||
succeedingCharacter == ' ' || succeedingCharacter == '\n' || succeedingCharacter == 0
succeedingCharacter == ' ' || succeedingCharacter == '\n' || succeedingCharacter == 0 ||
succeedingCharacter == '$' ||
isInMarkdownLinkText(block, line[i+len(stopMark):])
if checkSurrounding && !isValidEndingChar {
break
}

View File

@@ -10,9 +10,12 @@ import (
// ReleaseAsset represents a release asset
type ReleaseAsset struct {
ID int64
Name string
ContentType *string `yaml:"content_type"`
ID int64
Name string
// There was a field "ContentType (content_type)" because Some forges can provide that for assets,
// but we don't need it when migrating, so the field is omitted here.
Size *int
DownloadCount *int `yaml:"download_count"`
Created time.Time

View File

@@ -34,7 +34,7 @@ var (
var (
fieldPattern = regexp.MustCompile(`\A\S+:`)
namePattern = regexp.MustCompile(`\A[a-zA-Z][a-zA-Z0-9\.]*[a-zA-Z0-9]\z`)
versionPattern = regexp.MustCompile(`\A[0-9]+(?:[.\-][0-9]+){1,3}\z`)
versionPattern = regexp.MustCompile(`\A[0-9]+(?:[.\-][0-9]+)+\z`)
authorReplacePattern = regexp.MustCompile(`[\[\(].+?[\]\)]`)
)

View File

@@ -128,13 +128,22 @@ func TestParseDescription(t *testing.T) {
})
t.Run("InvalidVersion", func(t *testing.T) {
for _, version := range []string{"1", "1 0", "1.2.3.4.5", "1-2-3-4-5", "1.", "1.0.", "1-", "1-0-"} {
for _, version := range []string{"1", "1 0", "1.", "1.0.", "1-", "1-0-"} {
p, err := ParseDescription(createDescription(packageName, version))
assert.Nil(t, p)
assert.ErrorIs(t, err, ErrInvalidVersion)
}
})
t.Run("ValidVersionManyComponents", func(t *testing.T) {
for _, version := range []string{"0.3.4.0.2", "1.2.3.4.5", "1-2-3-4-5"} {
p, err := ParseDescription(createDescription(packageName, version))
assert.NoError(t, err)
assert.NotNil(t, p)
assert.Equal(t, version, p.Version)
}
})
t.Run("Valid", func(t *testing.T) {
p, err := ParseDescription(createDescription(packageName, packageVersion))
assert.NoError(t, err)

View File

@@ -62,7 +62,28 @@ type PackageMetadata struct {
Author User `json:"author"`
ReadmeFilename string `json:"readmeFilename,omitempty"`
Users map[string]bool `json:"users,omitempty"`
License string `json:"license,omitempty"`
License License `json:"license,omitempty"`
}
type License string
func (l *License) UnmarshalJSON(data []byte) error {
switch data[0] {
case '"':
var value string
if err := json.Unmarshal(data, &value); err != nil {
return err
}
*l = License(value)
case '{':
var values map[string]any
if err := json.Unmarshal(data, &values); err != nil {
return err
}
value, _ := values["type"].(string)
*l = License(value)
}
return nil
}
// PackageMetadataVersion documentation: https://github.com/npm/registry/blob/master/docs/REGISTRY-API.md#version
@@ -74,7 +95,7 @@ type PackageMetadataVersion struct {
Description string `json:"description"`
Author User `json:"author"`
Homepage string `json:"homepage,omitempty"`
License string `json:"license,omitempty"`
License License `json:"license,omitempty"`
Repository Repository `json:"repository"`
Keywords []string `json:"keywords,omitempty"`
Dependencies map[string]string `json:"dependencies,omitempty"`

View File

@@ -13,6 +13,7 @@ import (
"code.gitea.io/gitea/modules/json"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestParsePackage(t *testing.T) {
@@ -291,11 +292,36 @@ func TestParsePackage(t *testing.T) {
assert.Equal(t, packageDescription, p.Metadata.Readme)
assert.Equal(t, packageAuthor, p.Metadata.Author)
assert.Equal(t, packageBin, p.Metadata.Bin["bin"])
assert.Equal(t, "MIT", p.Metadata.License)
assert.Equal(t, "MIT", string(p.Metadata.License))
assert.Equal(t, "https://gitea.io/", p.Metadata.ProjectURL)
assert.Contains(t, p.Metadata.Dependencies, "package")
assert.Equal(t, "1.2.0", p.Metadata.Dependencies["package"])
assert.Equal(t, repository.Type, p.Metadata.Repository.Type)
assert.Equal(t, repository.URL, p.Metadata.Repository.URL)
})
t.Run("ValidLicenseMap", func(t *testing.T) {
packageJSON := `{
"versions": {
"0.1.1": {
"name": "dev-null",
"version": "0.1.1",
"license": {
"type": "MIT"
},
"dist": {
"integrity": "sha256-"
}
}
},
"_attachments": {
"foo": {
"data": "AAAA"
}
}
}`
p, err := ParsePackage(strings.NewReader(packageJSON))
require.NoError(t, err)
require.Equal(t, "MIT", string(p.Metadata.License))
})
}

View File

@@ -12,7 +12,7 @@ type Metadata struct {
Name string `json:"name,omitempty"`
Description string `json:"description,omitempty"`
Author string `json:"author,omitempty"`
License string `json:"license,omitempty"`
License License `json:"license,omitempty"`
ProjectURL string `json:"project_url,omitempty"`
Keywords []string `json:"keywords,omitempty"`
Dependencies map[string]string `json:"dependencies,omitempty"`

View File

@@ -233,7 +233,7 @@ func SyncReleasesWithTags(ctx context.Context, repo *repo_model.Repository, gitR
return fmt.Errorf("unable to update tag %s for pull-mirror Repo[%d:%s/%s]: %w", tag.Name, repo.ID, repo.OwnerName, repo.Name, err)
}
}
added, deleted, updated = len(deletes), len(updates), len(inserts)
added, deleted, updated = len(inserts), len(deletes), len(updates)
return nil
})
if err != nil {

View File

@@ -5,6 +5,7 @@ package setting
import (
"code.gitea.io/gitea/modules/container"
"code.gitea.io/gitea/modules/log"
)
// Admin settings
@@ -15,12 +16,33 @@ var Admin struct {
ExternalUserDisableFeatures container.Set[string]
}
var validUserFeatures = container.SetOf(
UserFeatureDeletion,
UserFeatureManageSSHKeys,
UserFeatureManageGPGKeys,
UserFeatureManageMFA,
UserFeatureManageCredentials,
UserFeatureChangeUsername,
UserFeatureChangeFullName,
)
func loadAdminFrom(rootCfg ConfigProvider) {
sec := rootCfg.Section("admin")
Admin.DisableRegularOrgCreation = sec.Key("DISABLE_REGULAR_ORG_CREATION").MustBool(false)
Admin.DefaultEmailNotification = sec.Key("DEFAULT_EMAIL_NOTIFICATIONS").MustString("enabled")
Admin.UserDisabledFeatures = container.SetOf(sec.Key("USER_DISABLED_FEATURES").Strings(",")...)
Admin.ExternalUserDisableFeatures = container.SetOf(sec.Key("EXTERNAL_USER_DISABLE_FEATURES").Strings(",")...).Union(Admin.UserDisabledFeatures)
for feature := range Admin.UserDisabledFeatures {
if !validUserFeatures.Contains(feature) {
log.Warn("USER_DISABLED_FEATURES contains unknown feature %q", feature)
}
}
for feature := range Admin.ExternalUserDisableFeatures {
if !validUserFeatures.Contains(feature) && !Admin.UserDisabledFeatures.Contains(feature) {
log.Warn("EXTERNAL_USER_DISABLE_FEATURES contains unknown feature %q", feature)
}
}
}
const (

View File

@@ -327,14 +327,14 @@ func LogStartupProblem(skip int, level log.Level, format string, args ...any) {
func deprecatedSetting(rootCfg ConfigProvider, oldSection, oldKey, newSection, newKey, version string) {
if rootCfg.Section(oldSection).HasKey(oldKey) {
LogStartupProblem(1, log.ERROR, "Deprecation: config option `[%s].%s` presents, please use `[%s].%s` instead because this fallback will be/has been removed in %s", oldSection, oldKey, newSection, newKey, version)
LogStartupProblem(1, log.ERROR, "Deprecation: config option `[%s].%s` present, please use `[%s].%s` instead because this fallback will be/has been removed in %s", oldSection, oldKey, newSection, newKey, version)
}
}
// deprecatedSettingDB add a hint that the configuration has been moved to database but still kept in app.ini
func deprecatedSettingDB(rootCfg ConfigProvider, oldSection, oldKey string) {
if rootCfg.Section(oldSection).HasKey(oldKey) {
LogStartupProblem(1, log.ERROR, "Deprecation: config option `[%s].%s` presents but it won't take effect because it has been moved to admin panel -> config setting", oldSection, oldKey)
LogStartupProblem(1, log.ERROR, "Deprecation: config option `[%s].%s` present but it won't take effect because it has been moved to admin panel -> config setting", oldSection, oldKey)
}
}

View File

@@ -5,6 +5,7 @@ package storage
import (
"context"
"errors"
"fmt"
"io"
"net/url"
@@ -27,25 +28,32 @@ type LocalStorage struct {
// NewLocalStorage returns a local files
func NewLocalStorage(ctx context.Context, config *setting.Storage) (ObjectStorage, error) {
// prepare storage root path
if !filepath.IsAbs(config.Path) {
return nil, fmt.Errorf("LocalStorageConfig.Path should have been prepared by setting/storage.go and should be an absolute path, but not: %q", config.Path)
}
log.Info("Creating new Local Storage at %s", config.Path)
if err := os.MkdirAll(config.Path, os.ModePerm); err != nil {
return nil, err
return nil, fmt.Errorf("LocalStorage config.Path should have been prepared by setting/storage.go and should be an absolute path, but not: %q", config.Path)
}
storageRoot := util.FilePathJoinAbs(config.Path)
if config.TemporaryPath == "" {
config.TemporaryPath = filepath.Join(config.Path, "tmp")
// prepare storage temporary path
storageTmp := config.TemporaryPath
if storageTmp == "" {
storageTmp = filepath.Join(storageRoot, "tmp")
}
if !filepath.IsAbs(config.TemporaryPath) {
return nil, fmt.Errorf("LocalStorageConfig.TemporaryPath should be an absolute path, but not: %q", config.TemporaryPath)
if !filepath.IsAbs(storageTmp) {
return nil, fmt.Errorf("LocalStorage config.TemporaryPath should be an absolute path, but not: %q", config.TemporaryPath)
}
storageTmp = util.FilePathJoinAbs(storageTmp)
// create the storage root if not exist
log.Info("Creating new Local Storage at %s", storageRoot)
if err := os.MkdirAll(storageRoot, os.ModePerm); err != nil {
return nil, err
}
return &LocalStorage{
ctx: ctx,
dir: config.Path,
tmpdir: config.TemporaryPath,
dir: storageRoot,
tmpdir: storageTmp,
}, nil
}
@@ -108,9 +116,21 @@ func (l *LocalStorage) Stat(path string) (os.FileInfo, error) {
return os.Stat(l.buildLocalPath(path))
}
// Delete delete a file
func (l *LocalStorage) deleteEmptyParentDirs(localFullPath string) {
for parent := filepath.Dir(localFullPath); len(parent) > len(l.dir); parent = filepath.Dir(parent) {
if err := os.Remove(parent); err != nil {
// since the target file has been deleted, parent dir error is not related to the file deletion itself.
break
}
}
}
// Delete deletes the file in storage and removes the empty parent directories (if possible)
func (l *LocalStorage) Delete(path string) error {
return util.Remove(l.buildLocalPath(path))
localFullPath := l.buildLocalPath(path)
err := util.Remove(localFullPath)
l.deleteEmptyParentDirs(localFullPath)
return err
}
// URL gets the redirect URL to a file
@@ -118,34 +138,38 @@ func (l *LocalStorage) URL(path, name, _ string, reqParams url.Values) (*url.URL
return nil, ErrURLNotSupported
}
func (l *LocalStorage) normalizeWalkError(err error) error {
if errors.Is(err, os.ErrNotExist) {
// ignore it because the file may be deleted during the walk, and we don't care about it
return nil
}
return err
}
// IterateObjects iterates across the objects in the local storage
func (l *LocalStorage) IterateObjects(dirName string, fn func(path string, obj Object) error) error {
dir := l.buildLocalPath(dirName)
return filepath.WalkDir(dir, func(path string, d os.DirEntry, err error) error {
if err != nil {
return filepath.WalkDir(dir, func(path string, d os.DirEntry, errWalk error) error {
if err := l.ctx.Err(); err != nil {
return err
}
select {
case <-l.ctx.Done():
return l.ctx.Err()
default:
if errWalk != nil {
return l.normalizeWalkError(errWalk)
}
if path == l.dir {
return nil
}
if d.IsDir() {
if path == l.dir || d.IsDir() {
return nil
}
relPath, err := filepath.Rel(l.dir, path)
if err != nil {
return err
return l.normalizeWalkError(err)
}
obj, err := os.Open(path)
if err != nil {
return err
return l.normalizeWalkError(err)
}
defer obj.Close()
return fn(relPath, obj)
return fn(filepath.ToSlash(relPath), obj)
})
}

View File

@@ -4,11 +4,14 @@
package storage
import (
"os"
"strings"
"testing"
"code.gitea.io/gitea/modules/setting"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestBuildLocalPath(t *testing.T) {
@@ -53,6 +56,49 @@ func TestBuildLocalPath(t *testing.T) {
}
}
func TestLocalStorageDelete(t *testing.T) {
rootDir := t.TempDir()
st, err := NewLocalStorage(t.Context(), &setting.Storage{Path: rootDir})
require.NoError(t, err)
assertExists := func(t *testing.T, path string, exists bool) {
_, err = os.Stat(rootDir + "/" + path)
if exists {
require.NoError(t, err)
} else {
require.ErrorIs(t, err, os.ErrNotExist)
}
}
_, err = st.Save("dir/sub1/1-a.txt", strings.NewReader(""), -1)
require.NoError(t, err)
_, err = st.Save("dir/sub1/1-b.txt", strings.NewReader(""), -1)
require.NoError(t, err)
_, err = st.Save("dir/sub2/2-a.txt", strings.NewReader(""), -1)
require.NoError(t, err)
assertExists(t, "dir/sub1/1-a.txt", true)
assertExists(t, "dir/sub1/1-b.txt", true)
assertExists(t, "dir/sub2/2-a.txt", true)
require.NoError(t, st.Delete("dir/sub1/1-a.txt"))
assertExists(t, "dir/sub1", true)
assertExists(t, "dir/sub1/1-a.txt", false)
assertExists(t, "dir/sub1/1-b.txt", true)
assertExists(t, "dir/sub2/2-a.txt", true)
require.NoError(t, st.Delete("dir/sub1/1-b.txt"))
assertExists(t, ".", true)
assertExists(t, "dir/sub1", false)
assertExists(t, "dir/sub1/1-a.txt", false)
assertExists(t, "dir/sub1/1-b.txt", false)
assertExists(t, "dir/sub2/2-a.txt", true)
require.NoError(t, st.Delete("dir/sub2/2-a.txt"))
assertExists(t, ".", true)
assertExists(t, "dir", false)
}
func TestLocalStorageIterator(t *testing.T) {
testStorageIterator(t, setting.LocalStorageType, &setting.Storage{Path: t.TempDir()})
}

View File

@@ -68,7 +68,12 @@ type ObjectStorage interface {
Stat(path string) (os.FileInfo, error)
Delete(path string) error
URL(path, name, method string, reqParams url.Values) (*url.URL, error)
IterateObjects(path string, iterator func(path string, obj Object) error) error
// IterateObjects calls the iterator function for each object in the storage with the given path as prefix
// The "fullPath" argument in callback is the full path in this storage.
// * IterateObjects("", ...): iterate all objects in this storage
// * IterateObjects("sub-path", ...): iterate all objects with "sub-path" as prefix in this storage, the "fullPath" will be like "sub-path/xxx"
IterateObjects(basePath string, iterator func(fullPath string, obj Object) error) error
}
// Copy copies a file from source ObjectStorage to dest ObjectStorage

View File

@@ -134,7 +134,7 @@ type CreateRepoOption struct {
// Whether the repository is private
Private bool `json:"private"`
// Label-Set to use
IssueLabels string `json:"issue_labels"`
IssueLabels string `json:"issue_labels" binding:"MaxSize(255)"`
// Whether the repository should be auto-initialized?
AutoInit bool `json:"auto_init"`
// Whether the repository is template
@@ -142,15 +142,15 @@ type CreateRepoOption struct {
// Gitignores to use
Gitignores string `json:"gitignores"`
// License to use
License string `json:"license"`
License string `json:"license" binding:"MaxSize(100)"`
// Readme of the repository to create
Readme string `json:"readme"`
Readme string `json:"readme" binding:"MaxSize(255)"`
// DefaultBranch of the repository (used when initializes and in template)
DefaultBranch string `json:"default_branch" binding:"GitRefName;MaxSize(100)"`
// TrustModel of the repository
// enum: default,collaborator,committer,collaboratorcommitter
TrustModel string `json:"trust_model"`
// ObjectFormatName of the underlying git repository
// ObjectFormatName of the underlying git repository, empty string for default (sha1)
// enum: sha1,sha256
ObjectFormatName string `json:"object_format_name" binding:"MaxSize(6)"`
}

View File

@@ -107,6 +107,17 @@ func detectFileTypeBox(data []byte) (brands []string, found bool) {
return brands, true
}
func isEmbeddedOpenType(data []byte) bool {
// https://www.w3.org/submissions/EOT
if len(data) < 80 {
return false
}
version := binary.LittleEndian.Uint32(data[8:]) // Actually this standard is abandoned (for IE6-IE11 only), there are only 3 versions defined
magic := binary.LittleEndian.Uint16(data[34:36]) // MagicNumber: 0x504C ("LP")
reserved := data[64:80] // Reserved 1-4 (each: unsigned long)
return (version == 0x00010000 || version == 0x00020001 || version == 0x00020002) && magic == 0x504C && bytes.Count(reserved, []byte{0}) == len(reserved)
}
// DetectContentType extends http.DetectContentType with more content types. Defaults to text/plain if input is empty.
func DetectContentType(data []byte) SniffedType {
if len(data) == 0 {
@@ -119,6 +130,18 @@ func DetectContentType(data []byte) SniffedType {
data = data[:SniffContentSize]
}
const typeMsFontObject = "application/vnd.ms-fontobject"
if ct == typeMsFontObject {
// Stupid Golang blindly detects any content with 34th-35th bytes being "LP" as font.
// If it is not really for ".eot" content, we try to detect it again by hiding the "LP", see the test for more details.
if isEmbeddedOpenType(data) {
return SniffedType{typeMsFontObject}
}
data = slices.Clone(data)
data[34] = 'l'
ct = http.DetectContentType(data)
}
vars := globalVars()
// SVG is unsupported by http.DetectContentType, https://github.com/golang/go/issues/15888
detectByHTML := strings.Contains(ct, "text/plain") || strings.Contains(ct, "text/html")

View File

@@ -6,6 +6,7 @@ package typesniffer
import (
"encoding/base64"
"encoding/hex"
"net/http"
"strings"
"testing"
@@ -154,3 +155,25 @@ func TestDetectContentTypeAvif(t *testing.T) {
st := DetectContentType(buf)
assert.Equal(t, MimeTypeImageAvif, st.contentType)
}
func TestDetectContentTypeIncorrectFont(t *testing.T) {
s := "Stupid Golang keep detecting 34th LP as font"
// They don't want to have any improvement to it: https://github.com/golang/go/issues/77172
golangDetected := http.DetectContentType([]byte(s))
assert.Equal(t, "application/vnd.ms-fontobject", golangDetected)
// We have to make our patch to make it work correctly
ourDetected := DetectContentType([]byte(s))
assert.Equal(t, "text/plain; charset=utf-8", ourDetected.contentType)
// For binary content, ensure it still detects as font. The content is from "opensans-regular.eot"
b := []byte{
0x3d, 0x30, 0x00, 0x00, 0x6b, 0x2f, 0x00, 0x00, 0x02, 0x00, 0x02, 0x00, 0x04, 0x00, 0x00, 0x00,
0x02, 0x0b, 0x06, 0x06, 0x03, 0x05, 0x04, 0x02, 0x02, 0x04, 0x01, 0x00, 0x90, 0x01, 0x00, 0x00,
0x04, 0x00, 0x4c, 0x50, 0xef, 0x02, 0x00, 0xe0, 0x5b, 0x20, 0x00, 0x40, 0x28, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x9f, 0x01, 0x00, 0x20, 0x00, 0x00, 0x00, 0x00, 0x63, 0xf4, 0x17, 0x14,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x12, 0x00, 0x4f, 0x00, 0x70, 0x00, 0x65, 0x00, 0x6e, 0x00, 0x20, 0x00, 0x53, 0x00,
}
assert.Equal(t, "application/vnd.ms-fontobject", http.DetectContentType(b))
assert.Equal(t, "application/vnd.ms-fontobject", DetectContentType(b).contentType)
}

View File

@@ -64,7 +64,7 @@ func PathJoinRelX(elem ...string) string {
return PathJoinRel(elems...)
}
const pathSeparator = string(os.PathSeparator)
const filepathSeparator = string(os.PathSeparator)
// FilePathJoinAbs joins the path elements into a single file path, each element is cleaned by filepath.Clean separately.
// All slashes/backslashes are converted to path separators before cleaning, the result only contains path separators.
@@ -75,30 +75,32 @@ const pathSeparator = string(os.PathSeparator)
// {`/foo`, ``, `bar`} => `/foo/bar`
// {`/foo`, `..`, `bar`} => `/foo/bar`
func FilePathJoinAbs(base string, sub ...string) string {
elems := make([]string, 1, len(sub)+1)
// POSIX filesystem can have `\` in file names. Windows: `\` and `/` are both used for path separators
// to keep the behavior consistent, we do not allow `\` in file names, replace all `\` with `/`
if isOSWindows() {
elems[0] = filepath.Clean(base)
} else {
elems[0] = filepath.Clean(strings.ReplaceAll(base, "\\", pathSeparator))
if !isOSWindows() {
base = strings.ReplaceAll(base, "\\", filepathSeparator)
}
if !filepath.IsAbs(elems[0]) {
// This shouldn't happen. If there is really necessary to pass in relative path, return the full path with filepath.Abs() instead
panic(fmt.Sprintf("FilePathJoinAbs: %q (for path %v) is not absolute, do not guess a relative path based on current working directory", elems[0], elems))
if !filepath.IsAbs(base) {
// This shouldn't happen. If it is really necessary to handle relative paths, use filepath.Abs() to get absolute paths first
panic(fmt.Sprintf("FilePathJoinAbs: %q (for path %v) is not absolute, do not guess a relative path based on current working directory", base, sub))
}
if len(sub) == 0 {
return filepath.Clean(base)
}
elems := make([]string, 1, len(sub)+1)
elems[0] = base
for _, s := range sub {
if s == "" {
continue
}
if isOSWindows() {
elems = append(elems, filepath.Clean(pathSeparator+s))
elems = append(elems, filepath.Clean(filepathSeparator+s))
} else {
elems = append(elems, filepath.Clean(pathSeparator+strings.ReplaceAll(s, "\\", pathSeparator)))
elems = append(elems, filepath.Clean(filepathSeparator+strings.ReplaceAll(s, "\\", filepathSeparator)))
}
}
// the elems[0] must be an absolute path, just join them together
// the elems[0] must be an absolute path, just join them together, and Join will also do Clean
return filepath.Join(elems...)
}
@@ -115,12 +117,72 @@ func IsDir(dir string) (bool, error) {
return false, err
}
func IsRegularFile(filePath string) (bool, error) {
f, err := os.Lstat(filePath)
if err == nil {
return f.Mode().IsRegular(), nil
var ErrNotRegularPathFile = errors.New("not a regular file")
// ReadRegularPathFile reads a file with given sub path in root dir.
// It returns error when the path is not a regular file, or any parent path is not a regular directory.
func ReadRegularPathFile(root, filePathIn string, limit int) ([]byte, error) {
pathFields := strings.Split(PathJoinRelX(filePathIn), "/")
targetPathBuilder := strings.Builder{}
targetPathBuilder.Grow(len(root) + len(filePathIn) + 2)
targetPathBuilder.WriteString(root)
targetPathString := root
for i, subPath := range pathFields {
targetPathBuilder.WriteByte(filepath.Separator)
targetPathBuilder.WriteString(subPath)
targetPathString = targetPathBuilder.String()
expectFile := i == len(pathFields)-1
st, err := os.Lstat(targetPathString)
if err != nil {
return nil, err
}
if expectFile && !st.Mode().IsRegular() || !expectFile && !st.Mode().IsDir() {
return nil, fmt.Errorf("%w: %s", ErrNotRegularPathFile, filePathIn)
}
}
return false, err
f, err := os.Open(targetPathString)
if err != nil {
return nil, err
}
defer f.Close()
return ReadWithLimit(f, limit)
}
// WriteRegularPathFile writes data to a file with given sub path in root dir, it creates parent directories if necessary.
// The file is created with fileMode, and the directories are created with dirMode.
// It returns error when the path already exists but is not a regular file, or any parent path is not a regular directory.
func WriteRegularPathFile(root, filePathIn string, data []byte, dirMode, fileMode os.FileMode) error {
pathFields := strings.Split(PathJoinRelX(filePathIn), "/")
targetPathBuilder := strings.Builder{}
targetPathBuilder.Grow(len(root) + len(filePathIn) + 2)
targetPathBuilder.WriteString(root)
targetPathString := root
for i, subPath := range pathFields {
targetPathBuilder.WriteByte(filepath.Separator)
targetPathBuilder.WriteString(subPath)
targetPathString = targetPathBuilder.String()
expectFile := i == len(pathFields)-1
st, err := os.Lstat(targetPathString)
if err == nil {
if expectFile && !st.Mode().IsRegular() || !expectFile && !st.Mode().IsDir() {
return fmt.Errorf("%w: %s", ErrNotRegularPathFile, filePathIn)
}
continue
}
if !os.IsNotExist(err) {
return err
}
if !expectFile {
if err = os.Mkdir(targetPathString, dirMode); err != nil {
return err
}
}
}
return os.WriteFile(targetPathString, data, fileMode)
}
// IsExist checks whether a file or directory exists.

View File

@@ -6,6 +6,7 @@ package util
import (
"net/url"
"os"
"path/filepath"
"runtime"
"testing"
@@ -230,3 +231,70 @@ func TestListDirRecursively(t *testing.T) {
require.NoError(t, err)
assert.ElementsMatch(t, []string{"d1/f-d1", "d1/s1/f-d1s1"}, res)
}
func TestReadWriteRegularPathFile(t *testing.T) {
const readLimit = 10000
tmpDir := t.TempDir()
rootDir := tmpDir + "/root"
_ = os.Mkdir(rootDir, 0o755)
_ = os.WriteFile(tmpDir+"/other-file", []byte("other-content"), 0o755)
_ = os.Mkdir(rootDir+"/real-dir", 0o755)
_ = os.WriteFile(rootDir+"/real-dir/real-file", []byte("dummy-content"), 0o644)
_ = os.Symlink(rootDir+"/real-dir", rootDir+"/link-dir")
_ = os.Symlink(rootDir+"/real-dir/real-file", rootDir+"/real-dir/link-file")
t.Run("Read", func(t *testing.T) {
content, err := os.ReadFile(filepath.Join(rootDir, "../other-file"))
require.NoError(t, err)
assert.Equal(t, "other-content", string(content))
content, err = ReadRegularPathFile(rootDir, "../other-file", readLimit)
require.ErrorIs(t, err, os.ErrNotExist)
assert.Empty(t, string(content))
content, err = ReadRegularPathFile(rootDir, "real-dir/real-file", readLimit)
require.NoError(t, err)
assert.Equal(t, "dummy-content", string(content))
_, err = ReadRegularPathFile(rootDir, "link-dir/real-file", readLimit)
require.ErrorIs(t, err, ErrNotRegularPathFile)
_, err = ReadRegularPathFile(rootDir, "real-dir/link-file", readLimit)
require.ErrorIs(t, err, ErrNotRegularPathFile)
_, err = ReadRegularPathFile(rootDir, "link-dir/link-file", readLimit)
require.ErrorIs(t, err, ErrNotRegularPathFile)
})
t.Run("Write", func(t *testing.T) {
assertFileContent := func(path, expected string) {
data, err := os.ReadFile(path)
if expected == "" {
assert.ErrorIs(t, err, os.ErrNotExist)
return
}
require.NoError(t, err)
assert.Equal(t, expected, string(data), "file content mismatch for %s", path)
}
err := WriteRegularPathFile(rootDir, "new-dir/new-file", []byte("new-content"), 0o755, 0o644)
require.NoError(t, err)
assertFileContent(rootDir+"/new-dir/new-file", "new-content")
err = WriteRegularPathFile(rootDir, "link-dir/real-file", []byte("new-content"), 0o755, 0o644)
require.ErrorIs(t, err, ErrNotRegularPathFile)
err = WriteRegularPathFile(rootDir, "link-dir/link-file", []byte("new-content"), 0o755, 0o644)
require.ErrorIs(t, err, ErrNotRegularPathFile)
err = WriteRegularPathFile(rootDir, "link-dir/new-file", []byte("new-content"), 0o755, 0o644)
require.ErrorIs(t, err, ErrNotRegularPathFile)
err = WriteRegularPathFile(rootDir, "real-dir/link-file", []byte("new-content"), 0o755, 0o644)
require.ErrorIs(t, err, ErrNotRegularPathFile)
err = WriteRegularPathFile(rootDir, "../other-file", []byte("new-content"), 0o755, 0o644)
require.NoError(t, err)
assertFileContent(rootDir+"/../other-file", "other-content")
assertFileContent(rootDir+"/other-file", "new-content")
err = WriteRegularPathFile(rootDir, "real-dir/real-file", []byte("changed-content"), 0o755, 0o644)
require.NoError(t, err)
assertFileContent(rootDir+"/real-dir/real-file", "changed-content")
})
}

View File

@@ -46,11 +46,15 @@ func RouterMockPoint(pointName string) func(next http.Handler) http.Handler {
//
// Then the mock function will be executed as a middleware at the mock point.
// It only takes effect in testing mode (setting.IsInTesting == true).
func RouteMock(pointName string, h any) {
func RouteMock(pointName string, h any) func() {
if _, ok := routeMockPoints[pointName]; !ok {
panic("route mock point not found: " + pointName)
}
old := routeMockPoints[pointName]
routeMockPoints[pointName] = toHandlerProvider(h)
return func() {
routeMockPoints[pointName] = old
}
}
// RouteMockReset resets all mock points (no mock anymore)

View File

@@ -55,7 +55,7 @@ func NewRouter() *Router {
// Use supports two middlewares
func (r *Router) Use(middlewares ...any) {
for _, m := range middlewares {
if m != nil {
if !isNilOrFuncNil(m) {
r.chiRouter.Use(toHandlerProvider(m))
}
}

View File

@@ -214,6 +214,7 @@ more = More
buttons.heading.tooltip = Add heading
buttons.bold.tooltip = Add bold text
buttons.italic.tooltip = Add italic text
buttons.strikethrough.tooltip = Add strikethrough text
buttons.quote.tooltip = Quote text
buttons.code.tooltip = Add code
buttons.link.tooltip = Add a link
@@ -833,6 +834,7 @@ add_new_openid = Add New OpenID URI
add_email = Add Email Address
add_openid = Add OpenID URI
add_email_confirmation_sent = A confirmation email has been sent to "%s". Please check your inbox within the next %s to confirm your email address.
email_primary_not_found = The selected email address could not be found.
add_email_success = The new email address has been added.
email_preference_set_success = Email preference has been set successfully.
add_openid_success = The new OpenID address has been added.
@@ -1857,6 +1859,7 @@ pulls.desc = Enable pull requests and code reviews.
pulls.new = New Pull Request
pulls.new.blocked_user = Cannot create pull request because you are blocked by the repository owner.
pulls.new.must_collaborator = You must be a collaborator to create pull request.
pulls.new.already_existed = A pull request between these branches already exists
pulls.edit.already_changed = Unable to save changes to the pull request. It appears the content has already been changed by another user. Please refresh the page and try editing again to avoid overwriting their changes.
pulls.view = View Pull Request
pulls.compare_changes = New Pull Request

View File

@@ -98,7 +98,7 @@
"eslint-plugin-wc": "3.0.1",
"globals": "16.4.0",
"happy-dom": "18.0.1",
"markdownlint-cli": "0.45.0",
"markdownlint-cli": "0.48.0",
"material-icon-theme": "5.27.0",
"nolyfill": "1.0.44",
"postcss-html": "1.8.0",
@@ -107,7 +107,7 @@
"stylelint-declaration-block-no-ignored-properties": "2.8.0",
"stylelint-declaration-strict-value": "1.10.11",
"stylelint-value-no-unknown-custom-properties": "6.0.1",
"svgo": "4.0.0",
"svgo": "4.0.1",
"typescript-eslint": "8.43.0",
"updates": "16.7.0",
"vite-string-plugin": "1.4.6",

225
pnpm-lock.yaml generated
View File

@@ -298,8 +298,8 @@ importers:
specifier: 18.0.1
version: 18.0.1
markdownlint-cli:
specifier: 0.45.0
version: 0.45.0
specifier: 0.48.0
version: 0.48.0
material-icon-theme:
specifier: 5.27.0
version: 5.27.0
@@ -325,8 +325,8 @@ importers:
specifier: 6.0.1
version: 6.0.1(stylelint@16.24.0(typescript@5.9.2))
svgo:
specifier: 4.0.0
version: 4.0.0
specifier: 4.0.1
version: 4.0.1
typescript-eslint:
specifier: 8.43.0
version: 8.43.0(eslint@9.35.0(jiti@2.5.1))(typescript@5.9.2)
@@ -1836,6 +1836,10 @@ packages:
balanced-match@2.0.0:
resolution: {integrity: sha512-1ugUSr8BHXRnK23KfuYS+gVMC3LB8QGH9W1iGtDPsNWoQbgtXSExkBu2aDR4epiGWZOjZsj6lDl/N/AqqTC3UA==}
balanced-match@4.0.4:
resolution: {integrity: sha512-BLrgEcRTwX2o6gGxGOCNyMvGSp35YofuYzw9h1IMTRmKqttAZZVU67bdb9Pr2vUHA8+j3i2tJfjO6C6+4myGTA==}
engines: {node: 18 || 20 || >=22}
base64-js@1.5.1:
resolution: {integrity: sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA==}
@@ -1855,6 +1859,10 @@ packages:
brace-expansion@2.0.2:
resolution: {integrity: sha512-Jt0vHyM+jmUBqojB7E1NIYadt0vI0Qxjxd2TErW94wDz+E2LAm5vKMXXwg6ZZBTHPuUlDgQHKXvjGBdfcF1ZDQ==}
brace-expansion@5.0.4:
resolution: {integrity: sha512-h+DEnpVvxmfVefa4jFbCf5HdH5YMDXRsmKflpf1pILZWRFlTbJpxeU55nJl4Smt5HQaGzg1o6RHFPJaOqnmBDg==}
engines: {node: 18 || 20 || >=22}
braces@3.0.3:
resolution: {integrity: sha512-yQbXgO/OSZVD2IsiLlro+7Hf6Q18EJrKSEsdoMzKePKXct3gvD8oLcOQdIzGupr5Fj+EDe8gO/lxc1BzfMpxvA==}
engines: {node: '>=8'}
@@ -2014,9 +2022,9 @@ packages:
resolution: {integrity: sha512-Vw8qHK3bZM9y/P10u3Vib8o/DdkvA2OtPtZvD871QKjy74Wj1WSKFILMPRPSdUSx5RFK1arlJzEtA4PkFgnbuA==}
engines: {node: '>=18'}
commander@13.1.0:
resolution: {integrity: sha512-/rFeCpNJQbhSZjGVwO9RFV3xPqbnERS8MmIQzCtD/zl6gpJuV/bMLuN92oG3F7d8oDEHHRrujSXNUr8fpjntKw==}
engines: {node: '>=18'}
commander@14.0.3:
resolution: {integrity: sha512-H+y0Jo/T1RZ9qPP4Eh1pkcQcLRglraJaSLoyOtHxu6AapkjWVCy2Sit1QQ4x3Dng8qDlSsZEet7g5Pq06MvTgw==}
engines: {node: '>=20'}
commander@2.20.3:
resolution: {integrity: sha512-GpVkmM8vF2vQUkj2LvZmD35JxeJOLCwJ9cUkugyk2nuhbv3+mJvpLYYt+0+USMxE+oj+ey/lJEnhZw75x/OMcQ==}
@@ -2104,6 +2112,10 @@ packages:
resolution: {integrity: sha512-0eW44TGN5SQXU1mWSkKwFstI/22X2bG1nYzZTYMAWjylYURhse752YgbE4Cx46AC+bAvI+/dYTPRk1LqSUnu6w==}
engines: {node: ^10 || ^12.20.0 || ^14.13.0 || >=15.0.0}
css-tree@3.2.1:
resolution: {integrity: sha512-X7sjQzceUhu1u7Y/ylrRZFU2FS6LRiFVp6rKLPg23y3x3c3DOKAwuXGDp+PAGjh6CSnCjYeAul8pcT8bAl+lSA==}
engines: {node: ^10 || ^12.20.0 || ^14.13.0 || >=15.0.0}
css-what@6.2.2:
resolution: {integrity: sha512-u/O3vwbptzhMs3L1fQE82ZSLHQQfto5gyZzwteVIEyeaY5Fc7R4dapF/BvRoSYFeqfBk4m0V1Vafq5Pjv25wvA==}
engines: {node: '>= 6'}
@@ -2308,8 +2320,17 @@ packages:
supports-color:
optional: true
decode-named-character-reference@1.2.0:
resolution: {integrity: sha512-c6fcElNV6ShtZXmsgNgFFV5tVX2PaV4g+MOAkb8eXHvn6sryJBrZa9r0zV6+dtTyoCKxtDy5tyQ5ZwQuidtd+Q==}
debug@4.4.3:
resolution: {integrity: sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==}
engines: {node: '>=6.0'}
peerDependencies:
supports-color: '*'
peerDependenciesMeta:
supports-color:
optional: true
decode-named-character-reference@1.3.0:
resolution: {integrity: sha512-GtpQYB283KrPp6nRw50q3U9/VfOutZOe103qlN7BPP6Ad27xYnOIWv4lPzo8HCAL+mMZofJ9KEy30fq6MfaK6Q==}
decode-uri-component@0.2.2:
resolution: {integrity: sha512-FqUYQ+8o158GyGTrMFJms9qh3CqTKvAqgqsTnkLI8sKu0028orqBhxNMFkFen0zGyg6epACD32pjVk58ngIErQ==}
@@ -2675,6 +2696,10 @@ packages:
resolution: {integrity: sha512-ca9pw9fomFcKPvFLXhBKUK90ZvGibiGOvRJNbjljY7s7uq/5YO4BOzcYtJqExdx99rF6aAcnRxHmcUHcz6sQsg==}
engines: {node: '>=0.10'}
esquery@1.7.0:
resolution: {integrity: sha512-Ap6G0WQwcU/LHsvLwON1fAQX9Zp0A2Y6Y/cJBl9r/JbW90Zyg4/zbG6zzKa2OTALELarYHmKu0GhpM5EO+7T0g==}
engines: {node: '>=0.10'}
esrecurse@4.3.0:
resolution: {integrity: sha512-KmfKL3b6G+RXvP8N1vr3Tq1kL/oCFgn2NYXEtqP8/L3pKapUA4G8cFVaoF3SU323CD4XypR/ffioHmkti6/Tag==}
engines: {node: '>=4.0'}
@@ -2835,6 +2860,10 @@ packages:
resolution: {integrity: sha512-QZjmEOC+IT1uk6Rx0sX22V6uHWVwbdbxf1faPqJ1QhLdGgsRGCZoyaQBm/piRdJy/D2um6hM1UP7ZEeQ4EkP+Q==}
engines: {node: '>=18'}
get-east-asian-width@1.5.0:
resolution: {integrity: sha512-CQ+bEO+Tva/qlmw24dCejulK5pMzVnUOFOijVogd3KQs07HnRIgp8TGipvCCRT06xeYEbpbgwaCxglFyiuIcmA==}
engines: {node: '>=18'}
get-set-props@0.2.0:
resolution: {integrity: sha512-YCmOj+4YAeEB5Dd9jfp6ETdejMet4zSxXjNkgaa4npBEKRI9uDOGB5MmAdAgi2OoFGAKshYhCbmLq2DS03CgVA==}
engines: {node: '>=18.0.0'}
@@ -2860,11 +2889,6 @@ packages:
resolution: {integrity: sha512-7Bv8RF0k6xjo7d4A/PxYLbUCfb6c+Vpd2/mB2yRDlew7Jb5hEXiCD9ibfO7wpk8i4sevK6DFny9h7EYbM3/sHg==}
hasBin: true
glob@11.0.3:
resolution: {integrity: sha512-2Nim7dha1KVkaiF4q6Dj+ngPPMdfvLJEOpZk/jKiUAkqKebpGAWQXAq9z1xu9HKu5lWfqw/FASuccEjyznjPaA==}
engines: {node: 20 || >=22}
hasBin: true
glob@7.2.3:
resolution: {integrity: sha512-nFR0zLpU2YCaRxwoCJvL6UvCH2JFyFVIvwTLsIf21AuHlMskA1hhTdk+LlYJtOlYt9v6dvszD2BGRqBL+iQK9Q==}
deprecated: Glob versions prior to v9 are no longer supported
@@ -3102,10 +3126,6 @@ packages:
jackspeak@3.4.3:
resolution: {integrity: sha512-OGlZQpz2yfahA/Rd1Y8Cd9SIEsqvXkLVoSw/cgwhnhFMDbsQFeZYoJJ7bIZBS9BcamUW96asq/npPWugM+RQBw==}
jackspeak@4.1.1:
resolution: {integrity: sha512-zptv57P3GpL+O0I7VdMJNBZCu+BPHVQUk55Ft8/QCJjTVxrnJHuVuX/0Bl2A6/+2oyR/ZMEuFKwmzqqZ/U5nPQ==}
engines: {node: 20 || >=22}
jest-worker@27.5.1:
resolution: {integrity: sha512-7vuh85V5cdDofPyxn58nrPjBktZo0u9x1g8WtjQol+jZDaE+fhN+cIvTj11GndBnMnyfrUOG1sZQxCdjKh+DKg==}
engines: {node: '>= 10.13.0'}
@@ -3138,6 +3158,10 @@ packages:
resolution: {integrity: sha512-wpxZs9NoxZaJESJGIZTyDEaYpl0FKSA+FB9aJiyemKhMwkxQg63h4T1KJgUGHpTqPDNRcmmYLugrRjJlBtWvRA==}
hasBin: true
js-yaml@4.1.1:
resolution: {integrity: sha512-qQKT4zQxXl8lLwBtHMWwaTcGfFOZviOJet3Oy/xmGk2gZH677CJM9EvtfdSkgWcATZhj/55JZ0rmy3myCT5lsA==}
hasBin: true
jsdoc-type-pratt-parser@4.8.0:
resolution: {integrity: sha512-iZ8Bdb84lWRuGHamRXFyML07r21pcwBrLkHEuHgEY5UbCouBwv7ECknDRKzsQIXMiqpPymqtIf8TC/shYKB5rw==}
engines: {node: '>=12.0.0'}
@@ -3331,10 +3355,6 @@ packages:
lru-cache@10.4.3:
resolution: {integrity: sha512-JNAzZcXrCt42VGLuYz0zfAzDfAvJWW6AfYlDBQyDV5DClI2m5sAmK+OIO7s59XfsRsWHp02jAJrRadPRGTt6SQ==}
lru-cache@11.2.1:
resolution: {integrity: sha512-r8LA6i4LP4EeWOhqBaZZjDWwehd1xUJPCJd9Sv300H0ZmcUER4+JPh7bqqZeqs1o5pgtgvXm+d9UGrB5zZGDiQ==}
engines: {node: 20 || >=22}
magic-string@0.25.9:
resolution: {integrity: sha512-RmF0AsMzgt25qzqqLc1+MbHmhdx0ojF2Fvs4XnOqz2ZOBXzzkEwc/dJQZCYHAn7v1jbVOjAZfK8msRn4BxO4VQ==}
@@ -3344,17 +3364,17 @@ packages:
markdown-escape@2.0.0:
resolution: {integrity: sha512-Trz4v0+XWlwy68LJIyw3bLbsJiC8XAbRCKF9DbEtZjyndKOGVx6n+wNB0VfoRmY2LKboQLeniap3xrb6LGSJ8A==}
markdown-it@14.1.0:
resolution: {integrity: sha512-a54IwgWPaeBCAAsv13YgmALOF1elABB08FxO9i+r4VFk5Vl4pKokRPeX8u5TCgSsPi6ec1otfLjdOpVcgbpshg==}
markdown-it@14.1.1:
resolution: {integrity: sha512-BuU2qnTti9YKgK5N+IeMubp14ZUKUUw7yeJbkjtosvHiP0AZ5c8IAgEMk79D0eC8F23r4Ac/q8cAIFdm2FtyoA==}
hasBin: true
markdownlint-cli@0.45.0:
resolution: {integrity: sha512-GiWr7GfJLVfcopL3t3pLumXCYs8sgWppjIA1F/Cc3zIMgD3tmkpyZ1xkm1Tej8mw53B93JsDjgA3KOftuYcfOw==}
markdownlint-cli@0.48.0:
resolution: {integrity: sha512-NkZQNu2E0Q5qLEEHwWj674eYISTLD4jMHkBzDobujXd1kv+yCxi8jOaD/rZoQNW1FBBMMGQpuW5So8B51N/e0A==}
engines: {node: '>=20'}
hasBin: true
markdownlint@0.38.0:
resolution: {integrity: sha512-xaSxkaU7wY/0852zGApM8LdlIfGCW8ETZ0Rr62IQtAnUMlMuifsg09vWJcNYeL4f0anvr8Vo4ZQar8jGpV0btQ==}
markdownlint@0.40.0:
resolution: {integrity: sha512-UKybllYNheWac61Ia7T6fzuQNDZimFIpCg2w6hHjgV1Qu0w1TV0LlSgryUGzM0bkKQCBhy2FDhEELB73Kb0kAg==}
engines: {node: '>=20'}
marked@15.0.12:
@@ -3380,6 +3400,9 @@ packages:
mdn-data@2.12.2:
resolution: {integrity: sha512-IEn+pegP1aManZuckezWCO+XZQDplx1366JoVhTpMpBB1sPey/SbveZQUosKiKiGYjg1wH4pMlNgXbCiYgihQA==}
mdn-data@2.27.1:
resolution: {integrity: sha512-9Yubnt3e8A0OKwxYSXyhLymGW4sCufcLG6VdiDdUGVkPhpqLxlvP5vl1983gQjJl3tqbrM731mjaZaP68AgosQ==}
mdurl@2.0.0:
resolution: {integrity: sha512-Lf+9+2r+Tdp5wXDXC4PcIBjTDtq4UKjCPMQhKIuzpJNW0b96kVqSwW0bT7FhRSfmAiFYgP+SCRvdrDozfh0U5w==}
@@ -3494,6 +3517,10 @@ packages:
resolution: {integrity: sha512-IPZ167aShDZZUMdRk66cyQAW3qr0WzbHkPdMYa8bzZhlHhO3jALbKdxcaak7W9FfT2rZNpQuUu4Od7ILEpXSaw==}
engines: {node: 20 || >=22}
minimatch@10.2.4:
resolution: {integrity: sha512-oRjTw/97aTBN0RHbYCdtF1MQfvusSIBQM0IZEgzl6426+8jSC0nF1a/GmnVLpfB9yyr6g6FTqWqiZVbxrtaCIg==}
engines: {node: 18 || 20 || >=22}
minimatch@3.1.2:
resolution: {integrity: sha512-J7p63hRiAjw1NDEww1W7i37+ByIrOWO5XQQAzZ3VOcL0PNybwpfmV/N05zFAzwQ9USyEcX6t3UO+K5aqBQOIHw==}
@@ -3676,10 +3703,6 @@ packages:
resolution: {integrity: sha512-Xa4Nw17FS9ApQFJ9umLiJS4orGjm7ZzwUrwamcGQuHSzDyth9boKDaycYdDcZDuqYATXw4HFXgaqWTctW/v1HA==}
engines: {node: '>=16 || 14 >=14.18'}
path-scurry@2.0.0:
resolution: {integrity: sha512-ypGJsmGtdXUOeM5u93TyeIEfEhM6s+ljAhrk5vAvSx8uyY/02OvrZnA0YNGUrPXfpJMgI1ODd3nwz8Npx4O4cg==}
engines: {node: 20 || >=22}
path-type@4.0.0:
resolution: {integrity: sha512-gDKb8aZMDeD/tZWs9P6+q0J9Mwkdl6xMV8TjnGP3qJVJ06bdMgkbBlLU8IdfOsIsFz2BW1rNVT3XuNEl8zPAvw==}
engines: {node: '>=8'}
@@ -3997,8 +4020,9 @@ packages:
sax@1.2.4:
resolution: {integrity: sha512-NqVDv9TpANUjFm0N8uM5GxL36UgKi9/atZw+x7YFnQ8ckwFGKrl4xX4yWtrey3UJm5nP1kUbnYgLopqWNSRhWw==}
sax@1.4.1:
resolution: {integrity: sha512-+aWOz7yVScEGoKNd4PA10LZ8sk0A/z5+nXQG5giUO5rprX9jgYsTdov9qCchZiPIZezbZH+jRut8nPodFAX4Jg==}
sax@1.5.0:
resolution: {integrity: sha512-21IYA3Q5cQf089Z6tgaUTr7lDAyzoTPx5HRtbhsME8Udispad8dC/+sziTNugOEx54ilvatQ9YCzl4KQLPcRHA==}
engines: {node: '>=11.0.0'}
schema-utils@4.3.2:
resolution: {integrity: sha512-Gn/JaSk/Mt9gYubxTtSn/QCV4em9mpAPiR1rqy/Ocu19u/G9J5WWdNoUT4SiV6mFC3y6cxyFcFwdzPM3FgxGAQ==}
@@ -4017,6 +4041,11 @@ packages:
engines: {node: '>=10'}
hasBin: true
semver@7.7.4:
resolution: {integrity: sha512-vFKC2IEtQnVhpT78h1Yp8wzwrf8CM+MzKMHGJZfBtzhZNycRFnXsHk6E5TxIkkMsgNS7mdX3AGB7x2QM2di4lA==}
engines: {node: '>=10'}
hasBin: true
serialize-javascript@6.0.2:
resolution: {integrity: sha512-Saa1xPByTTq2gdeFZYLLo+RFE35NHZkAbqZeWNd3BpzppeVisAqpDjcp8dyf6uIvEqJRd46jemmyA4iFIeVk8g==}
@@ -4061,8 +4090,8 @@ packages:
resolution: {integrity: sha512-qMCMfhY040cVHT43K9BFygqYbUPFZKHOg7K73mtTWJRb8pyP3fzf4Ixd5SzdEJQ6MRUg/WBnOLxghZtKKurENQ==}
engines: {node: '>=10'}
smol-toml@1.3.4:
resolution: {integrity: sha512-UOPtVuYkzYGee0Bd2Szz8d2G3RfMfJ2t3qVdZUAozZyAk+a0Sxa+QKix0YCwjL/A1RR0ar44nCxaoN9FxdJGwA==}
smol-toml@1.6.0:
resolution: {integrity: sha512-4zemZi0HvTnYwLfrpk/CF9LOd9Lt87kAt50GnqhMpyF9U3poDAP2+iukq2bZsO/ufegbYehBkqINbsWxj4l4cw==}
engines: {node: '>= 18'}
solid-js@1.9.9:
@@ -4143,6 +4172,10 @@ packages:
resolution: {integrity: sha512-tsaTIkKW9b4N+AEj+SVA+WhJzV7/zMhcSu78mLKWSk7cXMOSHsBKFWUs0fWwq8QyK3MgJBQRX6Gbi4kYbdvGkQ==}
engines: {node: '>=18'}
string-width@8.1.0:
resolution: {integrity: sha512-Kxl3KJGb/gxkaUMOjRsQ8IrXiGW75O4E3RPjFIINOVH8AMl2SQ/yWdTzWwF3FevIX9LcMAjJW+GRwAlAbTSXdg==}
engines: {node: '>=20'}
strip-ansi@6.0.1:
resolution: {integrity: sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A==}
engines: {node: '>=8'}
@@ -4151,6 +4184,10 @@ packages:
resolution: {integrity: sha512-gmBGslpoQJtgnMAvOVqGZpEz9dyoKTCzy2nfz/n8aIFhN/jCE/rCmcxabB6jOOHV+0WNnylOxaxBQPSvcWklhA==}
engines: {node: '>=12'}
strip-ansi@7.2.0:
resolution: {integrity: sha512-yDPMNjp4WyfYBkHnjIRLfca1i6KMyGCtsVgoKe/z1+6vukgaENdgGBZt+ZmKPc4gavvEZ5OgHfHdrazhgNyG7w==}
engines: {node: '>=12'}
strip-bom@3.0.0:
resolution: {integrity: sha512-vavAMRXOgBVNF6nyEEmL3DBK19iRpDcoIwW+swQ+CbGiu7lju6t+JklA1MHweoWtadgt4ISVUsXLyDq34ddcwA==}
engines: {node: '>=4'}
@@ -4235,8 +4272,8 @@ packages:
svg-tags@1.0.0:
resolution: {integrity: sha512-ovssysQTa+luh7A5Weu3Rta6FJlFBBbInjOh722LIt6klpU2/HtdUbszju/G4devcvk8PGt7FCLv5wftu3THUA==}
svgo@4.0.0:
resolution: {integrity: sha512-VvrHQ+9uniE+Mvx3+C9IEe/lWasXCU0nXMY2kZeLrHNICuRiC8uMPyM14UEaMOFA5mhyQqEkB02VoQ16n3DLaw==}
svgo@4.0.1:
resolution: {integrity: sha512-XDpWUOPC6FEibaLzjfe0ucaV0YrOjYotGJO1WpF0Zd+n6ZGEQUsSugaoLq9QkEZtAfQIxT42UChcssDVPP3+/w==}
engines: {node: '>=16'}
hasBin: true
@@ -6292,6 +6329,8 @@ snapshots:
balanced-match@2.0.0: {}
balanced-match@4.0.4: {}
base64-js@1.5.1: {}
big.js@5.2.2: {}
@@ -6309,6 +6348,10 @@ snapshots:
dependencies:
balanced-match: 1.0.2
brace-expansion@5.0.4:
dependencies:
balanced-match: 4.0.4
braces@3.0.3:
dependencies:
fill-range: 7.1.1
@@ -6464,7 +6507,7 @@ snapshots:
commander@12.1.0: {}
commander@13.1.0: {}
commander@14.0.3: {}
commander@2.20.3: {}
@@ -6548,6 +6591,11 @@ snapshots:
mdn-data: 2.12.2
source-map-js: 1.2.1
css-tree@3.2.1:
dependencies:
mdn-data: 2.27.1
source-map-js: 1.2.1
css-what@6.2.2: {}
css@3.0.0:
@@ -6764,7 +6812,11 @@ snapshots:
dependencies:
ms: 2.1.3
decode-named-character-reference@1.2.0:
debug@4.4.3:
dependencies:
ms: 2.1.3
decode-named-character-reference@1.3.0:
dependencies:
character-entities: 2.0.2
@@ -7264,6 +7316,10 @@ snapshots:
dependencies:
estraverse: 5.3.0
esquery@1.7.0:
dependencies:
estraverse: 5.3.0
esrecurse@4.3.0:
dependencies:
estraverse: 5.3.0
@@ -7402,6 +7458,8 @@ snapshots:
get-east-asian-width@1.4.0: {}
get-east-asian-width@1.5.0: {}
get-set-props@0.2.0: {}
get-source@2.0.12:
@@ -7432,15 +7490,6 @@ snapshots:
package-json-from-dist: 1.0.1
path-scurry: 1.11.1
glob@11.0.3:
dependencies:
foreground-child: 3.3.1
jackspeak: 4.1.1
minimatch: 10.0.3
minipass: 7.1.2
package-json-from-dist: 1.0.1
path-scurry: 2.0.0
glob@7.2.3:
dependencies:
fs.realpath: 1.0.0
@@ -7647,10 +7696,6 @@ snapshots:
optionalDependencies:
'@pkgjs/parseargs': 0.11.0
jackspeak@4.1.1:
dependencies:
'@isaacs/cliui': 8.0.2
jest-worker@27.5.1:
dependencies:
'@types/node': 24.3.1
@@ -7675,6 +7720,10 @@ snapshots:
dependencies:
argparse: 2.0.1
js-yaml@4.1.1:
dependencies:
argparse: 2.0.1
jsdoc-type-pratt-parser@4.8.0: {}
jsep@1.4.0: {}
@@ -7833,8 +7882,6 @@ snapshots:
lru-cache@10.4.3: {}
lru-cache@11.2.1: {}
magic-string@0.25.9:
dependencies:
sourcemap-codec: 1.4.8
@@ -7845,7 +7892,7 @@ snapshots:
markdown-escape@2.0.0: {}
markdown-it@14.1.0:
markdown-it@14.1.1:
dependencies:
argparse: 2.0.1
entities: 4.5.0
@@ -7854,23 +7901,24 @@ snapshots:
punycode.js: 2.3.1
uc.micro: 2.1.0
markdownlint-cli@0.45.0:
markdownlint-cli@0.48.0:
dependencies:
commander: 13.1.0
glob: 11.0.3
commander: 14.0.3
deep-extend: 0.6.0
ignore: 7.0.5
js-yaml: 4.1.0
js-yaml: 4.1.1
jsonc-parser: 3.3.1
jsonpointer: 5.0.1
markdown-it: 14.1.0
markdownlint: 0.38.0
minimatch: 10.0.3
markdown-it: 14.1.1
markdownlint: 0.40.0
minimatch: 10.2.4
run-con: 1.3.2
smol-toml: 1.3.4
smol-toml: 1.6.0
tinyglobby: 0.2.15
transitivePeerDependencies:
- supports-color
markdownlint@0.38.0:
markdownlint@0.40.0:
dependencies:
micromark: 4.0.2
micromark-core-commonmark: 2.0.3
@@ -7880,6 +7928,7 @@ snapshots:
micromark-extension-gfm-table: 2.1.1
micromark-extension-math: 3.1.0
micromark-util-types: 2.0.2
string-width: 8.1.0
transitivePeerDependencies:
- supports-color
@@ -7900,6 +7949,8 @@ snapshots:
mdn-data@2.12.2: {}
mdn-data@2.27.1: {}
mdurl@2.0.0: {}
meow@13.2.0: {}
@@ -7935,7 +7986,7 @@ snapshots:
micromark-core-commonmark@2.0.3:
dependencies:
decode-named-character-reference: 1.2.0
decode-named-character-reference: 1.3.0
devlop: 1.1.0
micromark-factory-destination: 2.0.1
micromark-factory-label: 2.0.1
@@ -8086,8 +8137,8 @@ snapshots:
micromark@4.0.2:
dependencies:
'@types/debug': 4.1.12
debug: 4.4.1
decode-named-character-reference: 1.2.0
debug: 4.4.3
decode-named-character-reference: 1.3.0
devlop: 1.1.0
micromark-core-commonmark: 2.0.3
micromark-factory-space: 2.0.1
@@ -8126,6 +8177,10 @@ snapshots:
dependencies:
'@isaacs/brace-expansion': 5.0.0
minimatch@10.2.4:
dependencies:
brace-expansion: 5.0.4
minimatch@3.1.2:
dependencies:
brace-expansion: 1.1.12
@@ -8266,7 +8321,7 @@ snapshots:
'@types/unist': 2.0.11
character-entities-legacy: 3.0.0
character-reference-invalid: 2.0.1
decode-named-character-reference: 1.2.0
decode-named-character-reference: 1.3.0
is-alphanumerical: 2.0.1
is-decimal: 2.0.1
is-hexadecimal: 2.0.1
@@ -8295,11 +8350,6 @@ snapshots:
lru-cache: 10.4.3
minipass: 7.1.2
path-scurry@2.0.0:
dependencies:
lru-cache: 11.2.1
minipass: 7.1.2
path-type@4.0.0: {}
pathe@2.0.3: {}
@@ -8594,7 +8644,7 @@ snapshots:
sax@1.2.4: {}
sax@1.4.1: {}
sax@1.5.0: {}
schema-utils@4.3.2:
dependencies:
@@ -8613,6 +8663,8 @@ snapshots:
semver@7.7.2: {}
semver@7.7.4: {}
serialize-javascript@6.0.2:
dependencies:
randombytes: 2.1.0
@@ -8649,7 +8701,7 @@ snapshots:
astral-regex: 2.0.0
is-fullwidth-code-point: 3.0.0
smol-toml@1.3.4: {}
smol-toml@1.6.0: {}
solid-js@1.9.9:
dependencies:
@@ -8735,6 +8787,11 @@ snapshots:
get-east-asian-width: 1.4.0
strip-ansi: 7.1.2
string-width@8.1.0:
dependencies:
get-east-asian-width: 1.5.0
strip-ansi: 7.2.0
strip-ansi@6.0.1:
dependencies:
ansi-regex: 5.0.1
@@ -8743,6 +8800,10 @@ snapshots:
dependencies:
ansi-regex: 6.2.2
strip-ansi@7.2.0:
dependencies:
ansi-regex: 6.2.2
strip-bom@3.0.0: {}
strip-indent@4.1.0: {}
@@ -8861,15 +8922,15 @@ snapshots:
svg-tags@1.0.0: {}
svgo@4.0.0:
svgo@4.0.1:
dependencies:
commander: 11.1.0
css-select: 5.2.2
css-tree: 3.1.0
css-tree: 3.2.1
css-what: 6.2.2
csso: 5.0.5
picocolors: 1.1.1
sax: 1.4.1
sax: 1.5.0
svgson@5.3.1:
dependencies:
@@ -9207,13 +9268,13 @@ snapshots:
vue-eslint-parser@10.2.0(eslint@9.35.0(jiti@2.5.1)):
dependencies:
debug: 4.4.1
debug: 4.4.3
eslint: 9.35.0(jiti@2.5.1)
eslint-scope: 8.4.0
eslint-visitor-keys: 4.2.1
espree: 10.4.0
esquery: 1.6.0
semver: 7.7.2
esquery: 1.7.0
semver: 7.7.4
transitivePeerDependencies:
- supports-color

View File

@@ -241,7 +241,7 @@ func (ar artifactRoutes) uploadArtifact(ctx *ArtifactContext) {
}
// get upload file size
fileRealTotalSize, contentLength := getUploadFileSize(ctx)
fileRealTotalSize := getUploadFileSize(ctx)
// get artifact retention days
expiredDays := setting.Actions.ArtifactRetentionDays
@@ -265,17 +265,17 @@ func (ar artifactRoutes) uploadArtifact(ctx *ArtifactContext) {
return
}
// save chunk to storage, if success, return chunk stotal size
// save chunk to storage, if success, return chunks total size
// if artifact is not gzip when uploading, chunksTotalSize == fileRealTotalSize
// if artifact is gzip when uploading, chunksTotalSize < fileRealTotalSize
chunksTotalSize, err := saveUploadChunk(ar.fs, ctx, artifact, contentLength, runID)
chunksTotalSize, err := saveUploadChunkV3GetTotalSize(ar.fs, ctx, artifact, runID)
if err != nil {
log.Error("Error save upload chunk: %v", err)
ctx.HTTPError(http.StatusInternalServerError, "Error save upload chunk")
return
}
// update artifact size if zero or not match, over write artifact size
// update artifact size if zero or not match, overwrite artifact size
if artifact.FileSize == 0 ||
artifact.FileCompressedSize == 0 ||
artifact.FileSize != fileRealTotalSize ||

View File

@@ -12,7 +12,7 @@ import (
"fmt"
"hash"
"io"
"path/filepath"
"path"
"sort"
"strings"
"time"
@@ -20,18 +20,73 @@ import (
"code.gitea.io/gitea/models/actions"
"code.gitea.io/gitea/models/db"
"code.gitea.io/gitea/modules/log"
"code.gitea.io/gitea/modules/setting"
"code.gitea.io/gitea/modules/storage"
)
func saveUploadChunkBase(st storage.ObjectStorage, ctx *ArtifactContext,
artifact *actions.ActionArtifact,
contentSize, runID, start, end, length int64, checkMd5 bool,
) (int64, error) {
type saveUploadChunkOptions struct {
start int64
end *int64
checkMd5 bool
}
func makeTmpPathNameV3(runID int64) string {
return fmt.Sprintf("tmp-upload/run-%d", runID)
}
func makeTmpPathNameV4(runID int64) string {
return fmt.Sprintf("tmp-upload/run-%d-v4", runID)
}
func makeChunkFilenameV3(runID, artifactID, start int64, endPtr *int64) string {
var end int64
if endPtr != nil {
end = *endPtr
}
return fmt.Sprintf("%d-%d-%d-%d.chunk", runID, artifactID, start, end)
}
func parseChunkFileItemV3(st storage.ObjectStorage, fpath string) (*chunkFileItem, error) {
baseName := path.Base(fpath)
if !strings.HasSuffix(baseName, ".chunk") {
return nil, errSkipChunkFile
}
var item chunkFileItem
var unusedRunID int64
if _, err := fmt.Sscanf(baseName, "%d-%d-%d-%d.chunk", &unusedRunID, &item.ArtifactID, &item.Start, &item.End); err != nil {
return nil, err
}
item.Path = fpath
if item.End == 0 {
fi, err := st.Stat(item.Path)
if err != nil {
return nil, err
}
item.Size = fi.Size()
item.End = item.Start + item.Size - 1
} else {
item.Size = item.End - item.Start + 1
}
return &item, nil
}
func saveUploadChunkV3(st storage.ObjectStorage, ctx *ArtifactContext, artifact *actions.ActionArtifact,
runID int64, opts saveUploadChunkOptions,
) (writtenSize int64, retErr error) {
// build chunk store path
storagePath := fmt.Sprintf("tmp%d/%d-%d-%d-%d.chunk", runID, runID, artifact.ID, start, end)
storagePath := fmt.Sprintf("%s/%s", makeTmpPathNameV3(runID), makeChunkFilenameV3(runID, artifact.ID, opts.start, opts.end))
// "end" is optional, so "contentSize=-1" means read until EOF
contentSize := int64(-1)
if opts.end != nil {
contentSize = *opts.end - opts.start + 1
}
var r io.Reader = ctx.Req.Body
var hasher hash.Hash
if checkMd5 {
if opts.checkMd5 {
// use io.TeeReader to avoid reading all body to md5 sum.
// it writes data to hasher after reading end
// if hash is not matched, delete the read-end result
@@ -41,76 +96,81 @@ func saveUploadChunkBase(st storage.ObjectStorage, ctx *ArtifactContext,
// save chunk to storage
writtenSize, err := st.Save(storagePath, r, contentSize)
if err != nil {
return -1, fmt.Errorf("save chunk to storage error: %v", err)
return 0, fmt.Errorf("save chunk to storage error: %v", err)
}
var checkErr error
if checkMd5 {
defer func() {
if retErr != nil {
if err := st.Delete(storagePath); err != nil {
log.Error("Error deleting chunk: %s, %v", storagePath, err)
}
}
}()
if contentSize != -1 && writtenSize != contentSize {
return writtenSize, fmt.Errorf("writtenSize %d does not match contentSize %d", writtenSize, contentSize)
}
if opts.checkMd5 {
// check md5
reqMd5String := ctx.Req.Header.Get(artifactXActionsResultsMD5Header)
chunkMd5String := base64.StdEncoding.EncodeToString(hasher.Sum(nil))
log.Info("[artifact] check chunk md5, sum: %s, header: %s", chunkMd5String, reqMd5String)
log.Debug("[artifact] check chunk md5, sum: %s, header: %s", chunkMd5String, reqMd5String)
// if md5 not match, delete the chunk
if reqMd5String != chunkMd5String {
checkErr = errors.New("md5 not match")
return writtenSize, errors.New("md5 not match")
}
}
if writtenSize != contentSize {
checkErr = errors.Join(checkErr, fmt.Errorf("writtenSize %d not match contentSize %d", writtenSize, contentSize))
}
if checkErr != nil {
if err := st.Delete(storagePath); err != nil {
log.Error("Error deleting chunk: %s, %v", storagePath, err)
}
return -1, checkErr
}
log.Info("[artifact] save chunk %s, size: %d, artifact id: %d, start: %d, end: %d",
storagePath, contentSize, artifact.ID, start, end)
// return chunk total size
return length, nil
log.Debug("[artifact] save chunk %s, size: %d, artifact id: %d, start: %d, size: %d", storagePath, writtenSize, artifact.ID, opts.start, contentSize)
return writtenSize, nil
}
func saveUploadChunk(st storage.ObjectStorage, ctx *ArtifactContext,
artifact *actions.ActionArtifact,
contentSize, runID int64,
) (int64, error) {
func saveUploadChunkV3GetTotalSize(st storage.ObjectStorage, ctx *ArtifactContext, artifact *actions.ActionArtifact, runID int64) (totalSize int64, _ error) {
// parse content-range header, format: bytes 0-1023/146515
contentRange := ctx.Req.Header.Get("Content-Range")
start, end, length := int64(0), int64(0), int64(0)
if _, err := fmt.Sscanf(contentRange, "bytes %d-%d/%d", &start, &end, &length); err != nil {
log.Warn("parse content range error: %v, content-range: %s", err, contentRange)
return -1, fmt.Errorf("parse content range error: %v", err)
var start, end int64
if _, err := fmt.Sscanf(contentRange, "bytes %d-%d/%d", &start, &end, &totalSize); err != nil {
return 0, fmt.Errorf("parse content range error: %v", err)
}
return saveUploadChunkBase(st, ctx, artifact, contentSize, runID, start, end, length, true)
_, err := saveUploadChunkV3(st, ctx, artifact, runID, saveUploadChunkOptions{start: start, end: &end, checkMd5: true})
if err != nil {
return 0, err
}
return totalSize, nil
}
func appendUploadChunk(st storage.ObjectStorage, ctx *ArtifactContext,
artifact *actions.ActionArtifact,
start, contentSize, runID int64,
) (int64, error) {
end := start + contentSize - 1
return saveUploadChunkBase(st, ctx, artifact, contentSize, runID, start, end, contentSize, false)
// Returns uploaded length
func appendUploadChunkV3(st storage.ObjectStorage, ctx *ArtifactContext, artifact *actions.ActionArtifact, runID, start int64) (int64, error) {
opts := saveUploadChunkOptions{start: start}
if ctx.Req.ContentLength > 0 {
end := start + ctx.Req.ContentLength - 1
opts.end = &end
}
return saveUploadChunkV3(st, ctx, artifact, runID, opts)
}
type chunkFileItem struct {
RunID int64
ArtifactID int64
Start int64
End int64
Path string
// these offset/size related fields might be missing when parsing, they will be filled in the listing functions
Size int64
Start int64
End int64 // inclusive: Size=10, Start=0, End=9
ChunkName string // v4 only
}
func listChunksByRunID(st storage.ObjectStorage, runID int64) (map[int64][]*chunkFileItem, error) {
storageDir := fmt.Sprintf("tmp%d", runID)
func listV3UnorderedChunksMapByRunID(st storage.ObjectStorage, runID int64) (map[int64][]*chunkFileItem, error) {
storageDir := makeTmpPathNameV3(runID)
var chunks []*chunkFileItem
if err := st.IterateObjects(storageDir, func(fpath string, obj storage.Object) error {
baseName := filepath.Base(fpath)
// when read chunks from storage, it only contains storage dir and basename,
// no matter the subdirectory setting in storage config
item := chunkFileItem{Path: storageDir + "/" + baseName}
if _, err := fmt.Sscanf(baseName, "%d-%d-%d-%d.chunk", &item.RunID, &item.ArtifactID, &item.Start, &item.End); err != nil {
return fmt.Errorf("parse content range error: %v", err)
item, err := parseChunkFileItemV3(st, fpath)
if errors.Is(err, errSkipChunkFile) {
return nil
} else if err != nil {
return fmt.Errorf("unable to parse chunk name: %v", fpath)
}
chunks = append(chunks, &item)
chunks = append(chunks, item)
return nil
}); err != nil {
return nil, err
@@ -123,52 +183,78 @@ func listChunksByRunID(st storage.ObjectStorage, runID int64) (map[int64][]*chun
return chunksMap, nil
}
func listChunksByRunIDV4(st storage.ObjectStorage, runID, artifactID int64, blist *BlockList) ([]*chunkFileItem, error) {
storageDir := fmt.Sprintf("tmpv4%d", runID)
var chunks []*chunkFileItem
chunkMap := map[string]*chunkFileItem{}
dummy := &chunkFileItem{}
for _, name := range blist.Latest {
chunkMap[name] = dummy
func listOrderedChunksForArtifact(st storage.ObjectStorage, runID, artifactID int64, blist *BlockList) ([]*chunkFileItem, error) {
emptyListAsError := func(chunks []*chunkFileItem) ([]*chunkFileItem, error) {
if len(chunks) == 0 {
return nil, fmt.Errorf("no chunk found for artifact id: %d", artifactID)
}
return chunks, nil
}
storageDir := makeTmpPathNameV4(runID)
var chunks []*chunkFileItem
var chunkMapV4 map[string]*chunkFileItem
if blist != nil {
// make a dummy map for quick lookup of chunk names, the values are nil now and will be filled after iterating storage objects
chunkMapV4 = map[string]*chunkFileItem{}
for _, name := range blist.Latest {
chunkMapV4[name] = nil
}
}
if err := st.IterateObjects(storageDir, func(fpath string, obj storage.Object) error {
baseName := filepath.Base(fpath)
if !strings.HasPrefix(baseName, "block-") {
item, err := parseChunkFileItemV4(st, artifactID, fpath)
if errors.Is(err, errSkipChunkFile) {
return nil
} else if err != nil {
return fmt.Errorf("unable to parse chunk name: %v", fpath)
}
// when read chunks from storage, it only contains storage dir and basename,
// no matter the subdirectory setting in storage config
item := chunkFileItem{Path: storageDir + "/" + baseName, ArtifactID: artifactID}
var size int64
var b64chunkName string
if _, err := fmt.Sscanf(baseName, "block-%d-%d-%s", &item.RunID, &size, &b64chunkName); err != nil {
return fmt.Errorf("parse content range error: %v", err)
}
rchunkName, err := base64.URLEncoding.DecodeString(b64chunkName)
if err != nil {
return fmt.Errorf("failed to parse chunkName: %v", err)
}
chunkName := string(rchunkName)
item.End = item.Start + size - 1
if _, ok := chunkMap[chunkName]; ok {
chunkMap[chunkName] = &item
// Single chunk upload with block id
if _, ok := chunkMapV4[item.ChunkName]; ok {
chunkMapV4[item.ChunkName] = item
} else if chunkMapV4 == nil {
if chunks != nil {
return errors.New("blockmap is required for chunks > 1")
}
chunks = []*chunkFileItem{item}
}
return nil
}); err != nil {
return nil, err
}
for i, name := range blist.Latest {
chunk, ok := chunkMap[name]
if !ok || chunk.Path == "" {
return nil, fmt.Errorf("missing Chunk (%d/%d): %s", i, len(blist.Latest), name)
if blist == nil && chunks == nil {
chunkUnorderedItemsMapV3, err := listV3UnorderedChunksMapByRunID(st, runID)
if err != nil {
return nil, err
}
chunks = append(chunks, chunk)
if i > 0 {
chunk.Start = chunkMap[blist.Latest[i-1]].End + 1
chunk.End += chunk.Start
chunks = chunkUnorderedItemsMapV3[artifactID]
sort.Slice(chunks, func(i, j int) bool {
return chunks[i].Start < chunks[j].Start
})
return emptyListAsError(chunks)
}
if len(chunks) == 0 && blist != nil {
for i, name := range blist.Latest {
chunk := chunkMapV4[name]
if chunk == nil {
return nil, fmt.Errorf("missing chunk (%d/%d): %s", i, len(blist.Latest), name)
}
chunks = append(chunks, chunk)
}
}
return chunks, nil
for i, chunk := range chunks {
if i == 0 {
chunk.End += chunk.Size - 1
} else {
chunk.Start = chunkMapV4[blist.Latest[i-1]].End + 1
chunk.End = chunk.Start + chunk.Size - 1
}
}
return emptyListAsError(chunks)
}
func mergeChunksForRun(ctx *ArtifactContext, st storage.ObjectStorage, runID int64, artifactName string) error {
@@ -181,13 +267,13 @@ func mergeChunksForRun(ctx *ArtifactContext, st storage.ObjectStorage, runID int
return err
}
// read all uploading chunks from storage
chunksMap, err := listChunksByRunID(st, runID)
unorderedChunksMap, err := listV3UnorderedChunksMapByRunID(st, runID)
if err != nil {
return err
}
// range db artifacts to merge chunks
for _, art := range artifacts {
chunks, ok := chunksMap[art.ID]
chunks, ok := unorderedChunksMap[art.ID]
if !ok {
log.Debug("artifact %d chunks not found", art.ID)
continue
@@ -239,12 +325,14 @@ func mergeChunksForArtifact(ctx *ArtifactContext, chunks []*chunkFileItem, st st
}
mergedReader := io.MultiReader(readers...)
shaPrefix := "sha256:"
var hash hash.Hash
var hashSha256 hash.Hash
if strings.HasPrefix(checksum, shaPrefix) {
hash = sha256.New()
hashSha256 = sha256.New()
} else if checksum != "" {
setting.PanicInDevOrTesting("unsupported checksum format: %s, will skip the checksum verification", checksum)
}
if hash != nil {
mergedReader = io.TeeReader(mergedReader, hash)
if hashSha256 != nil {
mergedReader = io.TeeReader(mergedReader, hashSha256)
}
// if chunk is gzip, use gz as extension
@@ -274,8 +362,8 @@ func mergeChunksForArtifact(ctx *ArtifactContext, chunks []*chunkFileItem, st st
}
}()
if hash != nil {
rawChecksum := hash.Sum(nil)
if hashSha256 != nil {
rawChecksum := hashSha256.Sum(nil)
actualChecksum := hex.EncodeToString(rawChecksum)
if !strings.HasSuffix(checksum, actualChecksum) {
return fmt.Errorf("update artifact error checksum is invalid %v vs %v", checksum, actualChecksum)

View File

@@ -20,8 +20,8 @@ const (
artifactXActionsResultsMD5Header = "x-actions-results-md5"
)
// The rules are from https://github.com/actions/toolkit/blob/main/packages/artifact/src/internal/path-and-artifact-name-validation.ts#L32
var invalidArtifactNameChars = strings.Join([]string{"\\", "/", "\"", ":", "<", ">", "|", "*", "?", "\r", "\n"}, "")
// The rules are from https://github.com/actions/toolkit/blob/main/packages/artifact/src/internal/upload/path-and-artifact-name-validation.ts
const invalidArtifactNameChars = "\\/\":<>|*?\r\n"
func validateArtifactName(ctx *ArtifactContext, artifactName string) bool {
if strings.ContainsAny(artifactName, invalidArtifactNameChars) {
@@ -84,11 +84,10 @@ func parseArtifactItemPath(ctx *ArtifactContext) (string, string, bool) {
// getUploadFileSize returns the size of the file to be uploaded.
// The raw size is the size of the file as reported by the header X-TFS-FileLength.
func getUploadFileSize(ctx *ArtifactContext) (int64, int64) {
contentLength := ctx.Req.ContentLength
func getUploadFileSize(ctx *ArtifactContext) int64 {
xTfsLength, _ := strconv.ParseInt(ctx.Req.Header.Get(artifactXTfsFileLengthHeader), 10, 64)
if xTfsLength > 0 {
return xTfsLength, contentLength
return xTfsLength
}
return contentLength, contentLength
return ctx.Req.ContentLength
}

View File

@@ -90,10 +90,12 @@ import (
"crypto/sha256"
"encoding/base64"
"encoding/xml"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"path"
"strconv"
"strings"
"time"
@@ -109,7 +111,7 @@ import (
"code.gitea.io/gitea/services/context"
"google.golang.org/protobuf/encoding/protojson"
protoreflect "google.golang.org/protobuf/reflect/protoreflect"
"google.golang.org/protobuf/reflect/protoreflect"
"google.golang.org/protobuf/types/known/timestamppb"
)
@@ -157,33 +159,81 @@ func ArtifactsV4Routes(prefix string) *web.Router {
return m
}
func (r artifactV4Routes) buildSignature(endp, expires, artifactName string, taskID, artifactID int64) []byte {
func (r *artifactV4Routes) buildSignature(endpoint, expires, artifactName string, taskID, artifactID int64) []byte {
mac := hmac.New(sha256.New, setting.GetGeneralTokenSigningSecret())
mac.Write([]byte(endp))
mac.Write([]byte(endpoint))
mac.Write([]byte(expires))
mac.Write([]byte(artifactName))
fmt.Fprint(mac, taskID)
fmt.Fprint(mac, artifactID)
_, _ = fmt.Fprint(mac, taskID)
_, _ = fmt.Fprint(mac, artifactID)
return mac.Sum(nil)
}
func (r artifactV4Routes) buildArtifactURL(ctx *ArtifactContext, endp, artifactName string, taskID, artifactID int64) string {
func (r *artifactV4Routes) buildArtifactURL(ctx *ArtifactContext, endpoint, artifactName string, taskID, artifactID int64) string {
expires := time.Now().Add(60 * time.Minute).Format("2006-01-02 15:04:05.999999999 -0700 MST")
uploadURL := strings.TrimSuffix(httplib.GuessCurrentAppURL(ctx), "/") + strings.TrimSuffix(r.prefix, "/") +
"/" + endp + "?sig=" + base64.URLEncoding.EncodeToString(r.buildSignature(endp, expires, artifactName, taskID, artifactID)) + "&expires=" + url.QueryEscape(expires) + "&artifactName=" + url.QueryEscape(artifactName) + "&taskID=" + strconv.FormatInt(taskID, 10) + "&artifactID=" + strconv.FormatInt(artifactID, 10)
"/" + endpoint +
"?sig=" + base64.RawURLEncoding.EncodeToString(r.buildSignature(endpoint, expires, artifactName, taskID, artifactID)) +
"&expires=" + url.QueryEscape(expires) +
"&artifactName=" + url.QueryEscape(artifactName) +
"&taskID=" + strconv.FormatInt(taskID, 10) +
"&artifactID=" + strconv.FormatInt(artifactID, 10)
return uploadURL
}
func (r artifactV4Routes) verifySignature(ctx *ArtifactContext, endp string) (*actions.ActionTask, string, bool) {
func makeBlockFilenameV4(runID, artifactID, size int64, blockID string) string {
sizeInName := max(size, 0) // do not use "-1" in filename
return fmt.Sprintf("block-%d-%d-%d-%s", runID, artifactID, sizeInName, base64.URLEncoding.EncodeToString([]byte(blockID)))
}
var errSkipChunkFile = errors.New("skip this chunk file")
func parseChunkFileItemV4(st storage.ObjectStorage, artifactID int64, fpath string) (*chunkFileItem, error) {
baseName := path.Base(fpath)
if !strings.HasPrefix(baseName, "block-") {
return nil, errSkipChunkFile
}
var item chunkFileItem
var unusedRunID int64
var b64chunkName string
_, err := fmt.Sscanf(baseName, "block-%d-%d-%d-%s", &unusedRunID, &item.ArtifactID, &item.Size, &b64chunkName)
if err != nil {
return nil, err
}
if item.ArtifactID != artifactID {
return nil, errSkipChunkFile
}
chunkName, err := base64.URLEncoding.DecodeString(b64chunkName)
if err != nil {
return nil, err
}
item.ChunkName = string(chunkName)
item.Path = fpath
if item.Size <= 0 {
fi, err := st.Stat(item.Path)
if err != nil {
return nil, err
}
item.Size = fi.Size()
}
return &item, nil
}
func (r *artifactV4Routes) verifySignature(ctx *ArtifactContext, endp string) (*actions.ActionTask, string, bool) {
rawTaskID := ctx.Req.URL.Query().Get("taskID")
rawArtifactID := ctx.Req.URL.Query().Get("artifactID")
sig := ctx.Req.URL.Query().Get("sig")
expires := ctx.Req.URL.Query().Get("expires")
artifactName := ctx.Req.URL.Query().Get("artifactName")
dsig, _ := base64.URLEncoding.DecodeString(sig)
taskID, _ := strconv.ParseInt(rawTaskID, 10, 64)
artifactID, _ := strconv.ParseInt(rawArtifactID, 10, 64)
dsig, errSig := base64.RawURLEncoding.DecodeString(sig)
taskID, errTask := strconv.ParseInt(rawTaskID, 10, 64)
artifactID, errArtifactID := strconv.ParseInt(rawArtifactID, 10, 64)
err := errors.Join(errSig, errTask, errArtifactID)
if err != nil {
log.Error("Error decoding signature values: %v", err)
ctx.HTTPError(http.StatusBadRequest, "Error decoding signature values")
return nil, "", false
}
expecedsig := r.buildSignature(endp, expires, artifactName, taskID, artifactID)
if !hmac.Equal(dsig, expecedsig) {
log.Error("Error unauthorized")
@@ -226,7 +276,7 @@ func (r *artifactV4Routes) getArtifactByName(ctx *ArtifactContext, runID int64,
return &art, nil
}
func (r *artifactV4Routes) parseProtbufBody(ctx *ArtifactContext, req protoreflect.ProtoMessage) bool {
func (r *artifactV4Routes) parseProtobufBody(ctx *ArtifactContext, req protoreflect.ProtoMessage) bool {
body, err := io.ReadAll(ctx.Req.Body)
if err != nil {
log.Error("Error decode request body: %v", err)
@@ -242,7 +292,7 @@ func (r *artifactV4Routes) parseProtbufBody(ctx *ArtifactContext, req protorefle
return true
}
func (r *artifactV4Routes) sendProtbufBody(ctx *ArtifactContext, req protoreflect.ProtoMessage) {
func (r *artifactV4Routes) sendProtobufBody(ctx *ArtifactContext, req protoreflect.ProtoMessage) {
resp, err := protojson.Marshal(req)
if err != nil {
log.Error("Error encode response body: %v", err)
@@ -257,7 +307,7 @@ func (r *artifactV4Routes) sendProtbufBody(ctx *ArtifactContext, req protoreflec
func (r *artifactV4Routes) createArtifact(ctx *ArtifactContext) {
var req CreateArtifactRequest
if ok := r.parseProtbufBody(ctx, &req); !ok {
if ok := r.parseProtobufBody(ctx, &req); !ok {
return
}
_, _, ok := validateRunIDV4(ctx, req.WorkflowRunBackendId)
@@ -291,7 +341,7 @@ func (r *artifactV4Routes) createArtifact(ctx *ArtifactContext) {
Ok: true,
SignedUploadUrl: r.buildArtifactURL(ctx, "UploadArtifact", artifactName, ctx.ActionTask.ID, artifact.ID),
}
r.sendProtbufBody(ctx, &respData)
r.sendProtobufBody(ctx, &respData)
}
func (r *artifactV4Routes) uploadArtifact(ctx *ArtifactContext) {
@@ -303,34 +353,34 @@ func (r *artifactV4Routes) uploadArtifact(ctx *ArtifactContext) {
comp := ctx.Req.URL.Query().Get("comp")
switch comp {
case "block", "appendBlock":
blockid := ctx.Req.URL.Query().Get("blockid")
if blockid == "" {
// get artifact by name
artifact, err := r.getArtifactByName(ctx, task.Job.RunID, artifactName)
// get artifact by name
artifact, err := r.getArtifactByName(ctx, task.Job.RunID, artifactName)
if err != nil {
log.Error("Error artifact not found: %v", err)
ctx.HTTPError(http.StatusNotFound, "Error artifact not found")
return
}
blockID := ctx.Req.URL.Query().Get("blockid")
if blockID == "" {
uploadedLength, err := appendUploadChunkV3(r.fs, ctx, artifact, artifact.RunID, artifact.FileSize)
if err != nil {
log.Error("Error artifact not found: %v", err)
ctx.HTTPError(http.StatusNotFound, "Error artifact not found")
log.Error("Error appending chunk %v", err)
ctx.HTTPError(http.StatusInternalServerError, "Error appending Chunk")
return
}
_, err = appendUploadChunk(r.fs, ctx, artifact, artifact.FileSize, ctx.Req.ContentLength, artifact.RunID)
if err != nil {
log.Error("Error runner api getting task: task is not running")
ctx.HTTPError(http.StatusInternalServerError, "Error runner api getting task: task is not running")
return
}
artifact.FileCompressedSize += ctx.Req.ContentLength
artifact.FileSize += ctx.Req.ContentLength
artifact.FileCompressedSize += uploadedLength
artifact.FileSize += uploadedLength
if err := actions.UpdateArtifactByID(ctx, artifact.ID, artifact); err != nil {
log.Error("Error UpdateArtifactByID: %v", err)
ctx.HTTPError(http.StatusInternalServerError, "Error UpdateArtifactByID")
return
}
} else {
_, err := r.fs.Save(fmt.Sprintf("tmpv4%d/block-%d-%d-%s", task.Job.RunID, task.Job.RunID, ctx.Req.ContentLength, base64.URLEncoding.EncodeToString([]byte(blockid))), ctx.Req.Body, -1)
blockFilename := makeBlockFilenameV4(task.Job.RunID, artifact.ID, ctx.Req.ContentLength, blockID)
_, err := r.fs.Save(fmt.Sprintf("%s/%s", makeTmpPathNameV4(task.Job.RunID), blockFilename), ctx.Req.Body, ctx.Req.ContentLength)
if err != nil {
log.Error("Error runner api getting task: task is not running")
ctx.HTTPError(http.StatusInternalServerError, "Error runner api getting task: task is not running")
log.Error("Error uploading block blob %v", err)
ctx.HTTPError(http.StatusInternalServerError, "Error uploading block blob")
return
}
}
@@ -338,10 +388,10 @@ func (r *artifactV4Routes) uploadArtifact(ctx *ArtifactContext) {
case "blocklist":
rawArtifactID := ctx.Req.URL.Query().Get("artifactID")
artifactID, _ := strconv.ParseInt(rawArtifactID, 10, 64)
_, err := r.fs.Save(fmt.Sprintf("tmpv4%d/%d-%d-blocklist", task.Job.RunID, task.Job.RunID, artifactID), ctx.Req.Body, -1)
_, err := r.fs.Save(fmt.Sprintf("%s/%d-%d-blocklist", makeTmpPathNameV4(task.Job.RunID), task.Job.RunID, artifactID), ctx.Req.Body, -1)
if err != nil {
log.Error("Error runner api getting task: task is not running")
ctx.HTTPError(http.StatusInternalServerError, "Error runner api getting task: task is not running")
log.Error("Error uploading blocklist %v", err)
ctx.HTTPError(http.StatusInternalServerError, "Error uploading blocklist")
return
}
ctx.JSON(http.StatusCreated, "created")
@@ -357,7 +407,7 @@ type Latest struct {
}
func (r *artifactV4Routes) readBlockList(runID, artifactID int64) (*BlockList, error) {
blockListName := fmt.Sprintf("tmpv4%d/%d-%d-blocklist", runID, runID, artifactID)
blockListName := fmt.Sprintf("%s/%d-%d-blocklist", makeTmpPathNameV4(runID), runID, artifactID)
s, err := r.fs.Open(blockListName)
if err != nil {
return nil, err
@@ -367,17 +417,22 @@ func (r *artifactV4Routes) readBlockList(runID, artifactID int64) (*BlockList, e
blockList := &BlockList{}
err = xdec.Decode(blockList)
_ = s.Close()
delerr := r.fs.Delete(blockListName)
if delerr != nil {
log.Warn("Failed to delete blockList %s: %v", blockListName, delerr)
}
return blockList, err
if err != nil {
return nil, err
}
return blockList, nil
}
func (r *artifactV4Routes) finalizeArtifact(ctx *ArtifactContext) {
var req FinalizeArtifactRequest
if ok := r.parseProtbufBody(ctx, &req); !ok {
if ok := r.parseProtobufBody(ctx, &req); !ok {
return
}
_, runID, ok := validateRunIDV4(ctx, req.WorkflowRunBackendId)
@@ -394,30 +449,20 @@ func (r *artifactV4Routes) finalizeArtifact(ctx *ArtifactContext) {
}
var chunks []*chunkFileItem
blockList, err := r.readBlockList(runID, artifact.ID)
blockList, blockListErr := r.readBlockList(runID, artifact.ID)
chunks, err = listOrderedChunksForArtifact(r.fs, runID, artifact.ID, blockList)
if err != nil {
log.Warn("Failed to read BlockList, fallback to old behavior: %v", err)
chunkMap, err := listChunksByRunID(r.fs, runID)
if err != nil {
log.Error("Error merge chunks: %v", err)
ctx.HTTPError(http.StatusInternalServerError, "Error merge chunks")
return
}
chunks, ok = chunkMap[artifact.ID]
if !ok {
log.Error("Error merge chunks")
ctx.HTTPError(http.StatusInternalServerError, "Error merge chunks")
return
}
} else {
chunks, err = listChunksByRunIDV4(r.fs, runID, artifact.ID, blockList)
if err != nil {
log.Error("Error merge chunks: %v", err)
ctx.HTTPError(http.StatusInternalServerError, "Error merge chunks")
return
}
artifact.FileSize = chunks[len(chunks)-1].End + 1
artifact.FileCompressedSize = chunks[len(chunks)-1].End + 1
log.Error("Error list chunks: %v", errors.Join(blockListErr, err))
ctx.HTTPError(http.StatusInternalServerError, "Error list chunks")
return
}
artifact.FileSize = chunks[len(chunks)-1].End + 1
artifact.FileCompressedSize = chunks[len(chunks)-1].End + 1
if req.Size != artifact.FileSize {
log.Error("Error merge chunks size mismatch")
ctx.HTTPError(http.StatusInternalServerError, "Error merge chunks size mismatch")
return
}
checksum := ""
@@ -434,13 +479,13 @@ func (r *artifactV4Routes) finalizeArtifact(ctx *ArtifactContext) {
Ok: true,
ArtifactId: artifact.ID,
}
r.sendProtbufBody(ctx, &respData)
r.sendProtobufBody(ctx, &respData)
}
func (r *artifactV4Routes) listArtifacts(ctx *ArtifactContext) {
var req ListArtifactsRequest
if ok := r.parseProtbufBody(ctx, &req); !ok {
if ok := r.parseProtobufBody(ctx, &req); !ok {
return
}
_, runID, ok := validateRunIDV4(ctx, req.WorkflowRunBackendId)
@@ -485,13 +530,13 @@ func (r *artifactV4Routes) listArtifacts(ctx *ArtifactContext) {
respData := ListArtifactsResponse{
Artifacts: list,
}
r.sendProtbufBody(ctx, &respData)
r.sendProtobufBody(ctx, &respData)
}
func (r *artifactV4Routes) getSignedArtifactURL(ctx *ArtifactContext) {
var req GetSignedArtifactURLRequest
if ok := r.parseProtbufBody(ctx, &req); !ok {
if ok := r.parseProtobufBody(ctx, &req); !ok {
return
}
_, runID, ok := validateRunIDV4(ctx, req.WorkflowRunBackendId)
@@ -525,7 +570,7 @@ func (r *artifactV4Routes) getSignedArtifactURL(ctx *ArtifactContext) {
if respData.SignedUrl == "" {
respData.SignedUrl = r.buildArtifactURL(ctx, "DownloadArtifact", artifactName, ctx.ActionTask.ID, artifact.ID)
}
r.sendProtbufBody(ctx, &respData)
r.sendProtobufBody(ctx, &respData)
}
func (r *artifactV4Routes) downloadArtifact(ctx *ArtifactContext) {
@@ -555,7 +600,7 @@ func (r *artifactV4Routes) downloadArtifact(ctx *ArtifactContext) {
func (r *artifactV4Routes) deleteArtifact(ctx *ArtifactContext) {
var req DeleteArtifactRequest
if ok := r.parseProtbufBody(ctx, &req); !ok {
if ok := r.parseProtobufBody(ctx, &req); !ok {
return
}
_, runID, ok := validateRunIDV4(ctx, req.WorkflowRunBackendId)
@@ -582,5 +627,5 @@ func (r *artifactV4Routes) deleteArtifact(ctx *ArtifactContext) {
Ok: true,
ArtifactId: artifact.ID,
}
r.sendProtbufBody(ctx, &respData)
r.sendProtobufBody(ctx, &respData)
}

View File

@@ -26,9 +26,18 @@ import (
// saveAsPackageBlob creates a package blob from an upload
// The uploaded blob gets stored in a special upload version to link them to the package/image
func saveAsPackageBlob(ctx context.Context, hsr packages_module.HashedSizeReader, pci *packages_service.PackageCreationInfo) (*packages_model.PackageBlob, error) { //nolint:unparam // PackageBlob is never used
// There will be concurrent uploading for the same blob, so it needs a global lock per blob hash
func saveAsPackageBlob(ctx context.Context, hsr packages_module.HashedSizeReader, pci *packages_service.PackageCreationInfo) (*packages_model.PackageBlob, error) { //nolint:unparam //returned PackageBlob is never used
pb := packages_service.NewPackageBlob(hsr)
err := globallock.LockAndDo(ctx, "container-blob:"+pb.HashSHA256, func(ctx context.Context) error {
var err error
pb, err = saveAsPackageBlobInternal(ctx, hsr, pci, pb)
return err
})
return pb, err
}
func saveAsPackageBlobInternal(ctx context.Context, hsr packages_module.HashedSizeReader, pci *packages_service.PackageCreationInfo, pb *packages_model.PackageBlob) (*packages_model.PackageBlob, error) {
exists := false
contentStore := packages_module.NewContentStore()
@@ -67,7 +76,7 @@ func saveAsPackageBlob(ctx context.Context, hsr packages_module.HashedSizeReader
return createFileForBlob(ctx, uploadVersion, pb)
})
if err != nil {
if !exists {
if !exists && pb != nil { // pb can be nil if GetOrInsertBlob failed
if err := contentStore.Delete(packages_module.BlobHash256Key(pb.HashSHA256)); err != nil {
log.Error("Error deleting package blob from content store: %v", err)
}

View File

@@ -290,8 +290,8 @@ func PostBlobsUploads(ctx *context.Context) {
Creator: ctx.Doer,
},
); err != nil {
switch err {
case packages_service.ErrQuotaTotalCount, packages_service.ErrQuotaTypeSize, packages_service.ErrQuotaTotalSize:
switch {
case errors.Is(err, packages_service.ErrQuotaTotalCount), errors.Is(err, packages_service.ErrQuotaTypeSize), errors.Is(err, packages_service.ErrQuotaTotalSize):
apiError(ctx, http.StatusForbidden, err)
default:
apiError(ctx, http.StatusInternalServerError, err)
@@ -439,8 +439,8 @@ func PutBlobsUpload(ctx *context.Context) {
Creator: ctx.Doer,
},
); err != nil {
switch err {
case packages_service.ErrQuotaTotalCount, packages_service.ErrQuotaTypeSize, packages_service.ErrQuotaTotalSize:
switch {
case errors.Is(err, packages_service.ErrQuotaTotalCount), errors.Is(err, packages_service.ErrQuotaTypeSize), errors.Is(err, packages_service.ErrQuotaTotalSize):
apiError(ctx, http.StatusForbidden, err)
default:
apiError(ctx, http.StatusInternalServerError, err)
@@ -592,13 +592,10 @@ func PutManifest(ctx *context.Context) {
apiErrorDefined(ctx, namedError)
} else if errors.Is(err, container_model.ErrContainerBlobNotExist) {
apiErrorDefined(ctx, errBlobUnknown)
} else if errors.Is(err, packages_service.ErrQuotaTotalCount) || errors.Is(err, packages_service.ErrQuotaTypeSize) || errors.Is(err, packages_service.ErrQuotaTotalSize) {
apiError(ctx, http.StatusForbidden, err)
} else {
switch err {
case packages_service.ErrQuotaTotalCount, packages_service.ErrQuotaTypeSize, packages_service.ErrQuotaTotalSize:
apiError(ctx, http.StatusForbidden, err)
default:
apiError(ctx, http.StatusInternalServerError, err)
}
apiError(ctx, http.StatusInternalServerError, err)
}
return
}

View File

@@ -82,9 +82,11 @@ type processManifestTxRet struct {
}
func handleCreateManifestResult(ctx context.Context, err error, mci *manifestCreationInfo, contentStore *packages_module.ContentStore, txRet *processManifestTxRet) (string, error) {
if err != nil && txRet.created && txRet.pb != nil {
if err := contentStore.Delete(packages_module.BlobHash256Key(txRet.pb.HashSHA256)); err != nil {
log.Error("Error deleting package blob from content store: %v", err)
if err != nil {
if txRet.created && txRet.pb != nil {
if err := contentStore.Delete(packages_module.BlobHash256Key(txRet.pb.HashSHA256)); err != nil {
log.Error("Error deleting package blob from content store: %v", err)
}
}
return "", err
}
@@ -198,14 +200,14 @@ func processOciImageIndex(ctx context.Context, mci *manifestCreationInfo, buf *p
if errors.Is(err, container_model.ErrContainerBlobNotExist) {
return errManifestBlobUnknown
}
return err
return fmt.Errorf("GetContainerBlob: %w", err)
}
size, err := packages_model.CalculateFileSize(ctx, &packages_model.PackageFileSearchOptions{
VersionID: pfd.File.VersionID,
})
if err != nil {
return err
return fmt.Errorf("CalculateFileSize: %w", err)
}
metadata.Manifests = append(metadata.Manifests, &container_module.Manifest{
@@ -217,7 +219,7 @@ func processOciImageIndex(ctx context.Context, mci *manifestCreationInfo, buf *p
pv, err := createPackageAndVersion(ctx, mci, metadata)
if err != nil {
return err
return fmt.Errorf("createPackageAndVersion: %w", err)
}
txRet.pv = pv
@@ -240,7 +242,7 @@ func createPackageAndVersion(ctx context.Context, mci *manifestCreationInfo, met
if p, err = packages_model.TryInsertPackage(ctx, p); err != nil {
if !errors.Is(err, packages_model.ErrDuplicatePackage) {
log.Error("Error inserting package: %v", err)
return nil, err
return nil, fmt.Errorf("TryInsertPackage: %w", err)
}
created = false
}
@@ -248,7 +250,7 @@ func createPackageAndVersion(ctx context.Context, mci *manifestCreationInfo, met
if created {
if _, err := packages_model.InsertProperty(ctx, packages_model.PropertyTypePackage, p.ID, container_module.PropertyRepository, strings.ToLower(mci.Owner.LowerName+"/"+mci.Image)); err != nil {
log.Error("Error setting package property: %v", err)
return nil, err
return nil, fmt.Errorf("InsertProperty(PropertyRepository): %w", err)
}
}
@@ -256,7 +258,7 @@ func createPackageAndVersion(ctx context.Context, mci *manifestCreationInfo, met
metadataJSON, err := json.Marshal(metadata)
if err != nil {
return nil, err
return nil, fmt.Errorf("json.Marshal(metadata): %w", err)
}
// "docker buildx imagetools create" multi-arch operations:
@@ -276,43 +278,43 @@ func createPackageAndVersion(ctx context.Context, mci *manifestCreationInfo, met
pv, err := packages_model.GetOrInsertVersion(ctx, _pv)
if err != nil {
if !errors.Is(err, packages_model.ErrDuplicatePackageVersion) {
log.Error("Error inserting package: %v", err)
return nil, err
log.Error("Error GetOrInsertVersion (first try) package: %v", err)
return nil, fmt.Errorf("GetOrInsertVersion: first try: %w", err)
}
if err = packages_service.DeletePackageVersionAndReferences(ctx, pv); err != nil {
return nil, err
return nil, fmt.Errorf("DeletePackageVersionAndReferences: %w", err)
}
// keep download count on overwriting
_pv.DownloadCount = pv.DownloadCount
pv, err = packages_model.GetOrInsertVersion(ctx, _pv)
if err != nil {
if !errors.Is(err, packages_model.ErrDuplicatePackageVersion) {
log.Error("Error inserting package: %v", err)
return nil, err
log.Error("Error GetOrInsertVersion (second try) package: %v", err)
return nil, fmt.Errorf("GetOrInsertVersion: second try: %w", err)
}
}
}
if err := packages_service.CheckCountQuotaExceeded(ctx, mci.Creator, mci.Owner); err != nil {
return nil, err
return nil, fmt.Errorf("CheckCountQuotaExceeded: %w", err)
}
if mci.IsTagged {
if err = packages_model.InsertOrUpdateProperty(ctx, packages_model.PropertyTypeVersion, pv.ID, container_module.PropertyManifestTagged, ""); err != nil {
return nil, err
return nil, fmt.Errorf("InsertOrUpdateProperty(ManifestTagged): %w", err)
}
} else {
if err = packages_model.DeletePropertiesByName(ctx, packages_model.PropertyTypeVersion, pv.ID, container_module.PropertyManifestTagged); err != nil {
return nil, err
return nil, fmt.Errorf("DeletePropertiesByName(ManifestTagged): %w", err)
}
}
if err = packages_model.DeletePropertiesByName(ctx, packages_model.PropertyTypeVersion, pv.ID, container_module.PropertyManifestReference); err != nil {
return nil, err
return nil, fmt.Errorf("DeletePropertiesByName(ManifestReference): %w", err)
}
for _, manifest := range metadata.Manifests {
if _, err = packages_model.InsertProperty(ctx, packages_model.PropertyTypeVersion, pv.ID, container_module.PropertyManifestReference, manifest.Digest); err != nil {
return nil, err
return nil, fmt.Errorf("InsertProperty(ManifestReference): %w", err)
}
}

View File

@@ -216,9 +216,12 @@ func EditUser(ctx *context.APIContext) {
}
if form.Email != nil {
if err := user_service.AdminAddOrSetPrimaryEmailAddress(ctx, ctx.ContextUser, *form.Email); err != nil {
if err := user_service.ReplacePrimaryEmailAddress(ctx, ctx.ContextUser, *form.Email); err != nil {
switch {
case user_model.IsErrEmailCharIsNotSupported(err), user_model.IsErrEmailInvalid(err):
if !user_model.IsEmailDomainAllowed(*form.Email) {
err = fmt.Errorf("the domain of user email %s conflicts with EMAIL_DOMAIN_ALLOWLIST or EMAIL_DOMAIN_BLOCKLIST", *form.Email)
}
ctx.APIError(http.StatusBadRequest, err)
case user_model.IsErrEmailAlreadyUsed(err):
ctx.APIError(http.StatusBadRequest, err)
@@ -227,10 +230,6 @@ func EditUser(ctx *context.APIContext) {
}
return
}
if !user_model.IsEmailDomainAllowed(*form.Email) {
ctx.Resp.Header().Add("X-Gitea-Warning", fmt.Sprintf("the domain of user email %s conflicts with EMAIL_DOMAIN_ALLOWLIST or EMAIL_DOMAIN_BLOCKLIST", *form.Email))
}
}
opts := &user_service.UpdateOptions{

View File

@@ -153,7 +153,7 @@ func repoAssignment() func(ctx *context.APIContext) {
if err != nil {
if user_model.IsErrUserNotExist(err) {
if redirectUserID, err := user_model.LookupUserRedirect(ctx, userName); err == nil {
context.RedirectToUser(ctx.Base, userName, redirectUserID)
context.RedirectToUser(ctx.Base, ctx.Doer, userName, redirectUserID)
} else if user_model.IsErrUserRedirectNotExist(err) {
ctx.APIErrorNotFound("GetUserByName", err)
} else {
@@ -629,7 +629,7 @@ func orgAssignment(args ...bool) func(ctx *context.APIContext) {
if organization.IsErrOrgNotExist(err) {
redirectUserID, err := user_model.LookupUserRedirect(ctx, ctx.PathParam("org"))
if err == nil {
context.RedirectToUser(ctx.Base, ctx.PathParam("org"), redirectUserID)
context.RedirectToUser(ctx.Base, ctx.Doer, ctx.PathParam("org"), redirectUserID)
} else if user_model.IsErrUserRedirectNotExist(err) {
ctx.APIErrorNotFound("GetOrgByName", err)
} else {
@@ -1400,19 +1400,19 @@ func Routes() *web.Router {
})
m.Get("/{base}/*", repo.GetPullRequestByBaseHead)
}, mustAllowPulls, reqRepoReader(unit.TypeCode), context.ReferencesGitRepo())
m.Group("/statuses", func() {
m.Group("/statuses", func() { // "/statuses/{sha}" only accepts commit ID
m.Combo("/{sha}").Get(repo.GetCommitStatuses).
Post(reqToken(), reqRepoWriter(unit.TypeCode), bind(api.CreateStatusOption{}), repo.NewCommitStatus)
}, reqRepoReader(unit.TypeCode))
m.Group("/commits", func() {
m.Get("", context.ReferencesGitRepo(), repo.GetAllCommits)
m.Group("/{ref}", func() {
m.Get("/status", repo.GetCombinedCommitStatusByRef)
m.Get("/statuses", repo.GetCommitStatusesByRef)
}, context.ReferencesGitRepo())
m.Group("/{sha}", func() {
m.Get("/pull", repo.GetCommitPullRequest)
}, context.ReferencesGitRepo())
m.PathGroup("/*", func(g *web.RouterPathGroup) {
// Mis-configured reverse proxy might decode the `%2F` to slash ahead, so we need to support both formats (escaped, unescaped) here.
// It also matches GitHub's behavior
g.MatchPath("GET", "/<ref:*>/status", repo.GetCombinedCommitStatusByRef)
g.MatchPath("GET", "/<ref:*>/statuses", repo.GetCommitStatusesByRef)
g.MatchPath("GET", "/<sha>/pull", repo.GetCommitPullRequest)
})
}, reqRepoReader(unit.TypeCode))
m.Group("/git", func() {
m.Group("/commits", func() {

View File

@@ -135,7 +135,7 @@ func GetUserOrgsPermissions(ctx *context.APIContext) {
op := api.OrganizationPermissions{}
if !organization.HasOrgOrUserVisible(ctx, o, ctx.ContextUser) {
if !organization.HasOrgOrUserVisible(ctx, o, ctx.Doer) {
ctx.APIErrorNotFound("HasOrgOrUserVisible", nil)
return
}

View File

@@ -224,7 +224,7 @@ func GetStopwatches(ctx *context.APIContext) {
return
}
apiSWs, err := convert.ToStopWatches(ctx, sws)
apiSWs, err := convert.ToStopWatches(ctx, ctx.Doer, sws)
if err != nil {
ctx.APIErrorInternal(err)
return

View File

@@ -356,7 +356,7 @@ func DeleteTime(ctx *context.APIContext) {
return
}
time, err := issues_model.GetTrackedTimeByID(ctx, ctx.PathParamInt64("id"))
time, err := issues_model.GetTrackedTimeByID(ctx, issue.ID, ctx.PathParamInt64("id"))
if err != nil {
if db.IsErrNotExist(err) {
ctx.APIErrorNotFound(err)

View File

@@ -29,6 +29,7 @@ import (
"code.gitea.io/gitea/modules/timeutil"
"code.gitea.io/gitea/modules/web"
"code.gitea.io/gitea/routers/api/v1/utils"
"code.gitea.io/gitea/routers/common"
asymkey_service "code.gitea.io/gitea/services/asymkey"
"code.gitea.io/gitea/services/automerge"
"code.gitea.io/gitea/services/context"
@@ -1076,7 +1077,7 @@ func parseCompareInfo(ctx *context.APIContext, form api.CreatePullRequestOption)
} else if len(headInfos) == 2 {
// There is a head repository (the head repository could also be the same base repo)
headRefToGuess = headInfos[1]
headUser, err = user_model.GetUserByName(ctx, headInfos[0])
headUser, err = user_model.GetUserOrOrgByName(ctx, headInfos[0])
if err != nil {
if user_model.IsErrUserNotExist(err) {
ctx.APIErrorNotFound("GetUserByName")
@@ -1092,28 +1093,23 @@ func parseCompareInfo(ctx *context.APIContext, form api.CreatePullRequestOption)
isSameRepo := ctx.Repo.Owner.ID == headUser.ID
// Check if current user has fork of repository or in the same repository.
headRepo := repo_model.GetForkedRepo(ctx, headUser.ID, baseRepo.ID)
if headRepo == nil && !isSameRepo {
err = baseRepo.GetBaseRepo(ctx)
var headRepo *repo_model.Repository
if isSameRepo {
headRepo = baseRepo
} else {
headRepo, err = common.FindHeadRepo(ctx, baseRepo, headUser.ID)
if err != nil {
ctx.APIErrorInternal(err)
return nil, nil
}
// Check if baseRepo's base repository is the same as headUser's repository.
if baseRepo.BaseRepo == nil || baseRepo.BaseRepo.OwnerID != headUser.ID {
log.Trace("parseCompareInfo[%d]: does not have fork or in same repository", baseRepo.ID)
ctx.APIErrorNotFound("GetBaseRepo")
if headRepo == nil {
ctx.APIErrorNotFound("head repository not found")
return nil, nil
}
// Assign headRepo so it can be used below.
headRepo = baseRepo.BaseRepo
}
var headGitRepo *git.Repository
if isSameRepo {
headRepo = ctx.Repo.Repository
headGitRepo = ctx.Repo.GitRepo
closer = func() {} // no need to close the head repo because it shares the base repo
} else {
@@ -1137,7 +1133,7 @@ func parseCompareInfo(ctx *context.APIContext, form api.CreatePullRequestOption)
return nil, nil
}
if !permBase.CanReadIssuesOrPulls(true) || !permBase.CanRead(unit.TypeCode) {
if !permBase.CanRead(unit.TypeCode) {
log.Trace("Permission Denied: User %-v cannot create/read pull requests or cannot read code in Repo %-v\nUser in baseRepo has Permissions: %-+v", ctx.Doer, baseRepo, permBase)
ctx.APIErrorNotFound("Can't read pulls or can't read UnitTypeCode")
return nil, nil
@@ -1341,7 +1337,7 @@ func CancelScheduledAutoMerge(ctx *context.APIContext) {
}
if ctx.Doer.ID != autoMerge.DoerID {
allowed, err := access_model.IsUserRepoAdmin(ctx, ctx.Repo.Repository, ctx.Doer)
allowed, err := pull_service.IsUserAllowedToMerge(ctx, pull, ctx.Repo.Permission, ctx.Doer)
if err != nil {
ctx.APIErrorInternal(err)
return
@@ -1552,9 +1548,9 @@ func GetPullRequestFiles(ctx *context.APIContext) {
var prInfo *pull_service.CompareInfo
if pr.HasMerged {
prInfo, err = pull_service.GetCompareInfo(ctx, pr.BaseRepo, pr.BaseRepo, baseGitRepo, pr.MergeBase, pr.GetGitHeadRefName(), true, false)
prInfo, err = pull_service.GetCompareInfo(ctx, pr.BaseRepo, pr.BaseRepo, baseGitRepo, pr.MergeBase, pr.GetGitHeadRefName(), false, false)
} else {
prInfo, err = pull_service.GetCompareInfo(ctx, pr.BaseRepo, pr.BaseRepo, baseGitRepo, pr.BaseBranch, pr.GetGitHeadRefName(), true, false)
prInfo, err = pull_service.GetCompareInfo(ctx, pr.BaseRepo, pr.BaseRepo, baseGitRepo, pr.BaseBranch, pr.GetGitHeadRefName(), false, false)
}
if err != nil {
ctx.APIErrorInternal(err)

View File

@@ -8,6 +8,7 @@ import (
"fmt"
"net/http"
auth_model "code.gitea.io/gitea/models/auth"
"code.gitea.io/gitea/models/db"
"code.gitea.io/gitea/models/perm"
repo_model "code.gitea.io/gitea/models/repo"
@@ -21,6 +22,28 @@ import (
release_service "code.gitea.io/gitea/services/release"
)
func hasRepoWriteScope(ctx *context.APIContext) bool {
scope, ok := ctx.Data["ApiTokenScope"].(auth_model.AccessTokenScope)
if ctx.Data["IsApiToken"] != true || !ok {
return true
}
requiredScopes := auth_model.GetRequiredScopes(auth_model.Write, auth_model.AccessTokenScopeCategoryRepository)
allow, err := scope.HasScope(requiredScopes...)
if err != nil {
ctx.APIError(http.StatusForbidden, "checking scope failed: "+err.Error())
return false
}
return allow
}
func canAccessDraftRelease(ctx *context.APIContext) bool {
if !ctx.IsSigned || !ctx.Repo.CanWrite(unit.TypeReleases) {
return false
}
return hasRepoWriteScope(ctx)
}
// GetRelease get a single release of a repository
func GetRelease(ctx *context.APIContext) {
// swagger:operation GET /repos/{owner}/{repo}/releases/{id} repository repoGetRelease
@@ -62,6 +85,15 @@ func GetRelease(ctx *context.APIContext) {
return
}
if release.IsDraft { // only the users with write access can see draft releases
if !canAccessDraftRelease(ctx) {
if !ctx.Written() {
ctx.APIErrorNotFound()
}
return
}
}
if err := release.LoadAttributes(ctx); err != nil {
ctx.APIErrorInternal(err)
return
@@ -151,9 +183,13 @@ func ListReleases(ctx *context.APIContext) {
// "$ref": "#/responses/notFound"
listOptions := utils.GetListOptions(ctx)
includeDrafts := (ctx.Repo.AccessMode >= perm.AccessModeWrite || ctx.Repo.UnitAccessMode(unit.TypeReleases) >= perm.AccessModeWrite) && hasRepoWriteScope(ctx)
if ctx.Written() {
return
}
opts := repo_model.FindReleasesOptions{
ListOptions: listOptions,
IncludeDrafts: ctx.Repo.AccessMode >= perm.AccessModeWrite || ctx.Repo.UnitAccessMode(unit.TypeReleases) >= perm.AccessModeWrite,
IncludeDrafts: includeDrafts,
IncludeTags: false,
IsDraft: ctx.FormOptionalBool("draft"),
IsPreRelease: ctx.FormOptionalBool("pre-release"),

View File

@@ -34,6 +34,14 @@ func checkReleaseMatchRepo(ctx *context.APIContext, releaseID int64) bool {
ctx.APIErrorNotFound()
return false
}
if release.IsDraft {
if !canAccessDraftRelease(ctx) {
if !ctx.Written() {
ctx.APIErrorNotFound()
}
return false
}
}
return true
}
@@ -141,6 +149,14 @@ func ListReleaseAttachments(ctx *context.APIContext) {
ctx.APIErrorNotFound()
return
}
if release.IsDraft {
if !canAccessDraftRelease(ctx) {
if !ctx.Written() {
ctx.APIErrorNotFound()
}
return
}
}
if err := release.LoadAttributes(ctx); err != nil {
ctx.APIErrorInternal(err)
return
@@ -398,7 +414,6 @@ func DeleteReleaseAttachment(ctx *context.APIContext) {
ctx.APIErrorNotFound()
return
}
// FIXME Should prove the existence of the given repo, but results in unnecessary database requests
if err := repo_model.DeleteAttachment(ctx, attach, true); err != nil {
ctx.APIErrorInternal(err)

View File

@@ -16,7 +16,7 @@ func GetUserByPathParam(ctx *context.APIContext, name string) *user_model.User {
if err != nil {
if user_model.IsErrUserNotExist(err) {
if redirectUserID, err2 := user_model.LookupUserRedirect(ctx, username); err2 == nil {
context.RedirectToUser(ctx.Base, username, redirectUserID)
context.RedirectToUser(ctx.Base, ctx.Doer, username, redirectUserID)
} else {
ctx.APIErrorNotFound("GetUserByName", err)
}

View File

@@ -44,9 +44,11 @@ func isRoutePathExpensive(routePattern string) bool {
"/{username}/{reponame}/blame/",
"/{username}/{reponame}/commit/",
"/{username}/{reponame}/commits/",
"/{username}/{reponame}/compare/",
"/{username}/{reponame}/graph",
"/{username}/{reponame}/media/",
"/{username}/{reponame}/raw/",
"/{username}/{reponame}/rss/branch/",
"/{username}/{reponame}/src/",
// issue & PR related (no trailing slash)

View File

@@ -4,6 +4,8 @@
package common
import (
"context"
repo_model "code.gitea.io/gitea/models/repo"
user_model "code.gitea.io/gitea/models/user"
"code.gitea.io/gitea/modules/git"
@@ -20,3 +22,54 @@ type CompareInfo struct {
HeadBranch string
DirectComparison bool
}
// maxForkTraverseLevel defines the maximum levels to traverse when searching for the head repository.
const maxForkTraverseLevel = 10
// FindHeadRepo tries to find the head repository based on the base repository and head user ID.
func FindHeadRepo(ctx context.Context, baseRepo *repo_model.Repository, headUserID int64) (*repo_model.Repository, error) {
if baseRepo.IsFork {
curRepo := baseRepo
for curRepo.OwnerID != headUserID { // We assume the fork deepth is not too deep.
if err := curRepo.GetBaseRepo(ctx); err != nil {
return nil, err
}
if curRepo.BaseRepo == nil {
return findHeadRepoFromRootBase(ctx, curRepo, headUserID, maxForkTraverseLevel)
}
curRepo = curRepo.BaseRepo
}
return curRepo, nil
}
return findHeadRepoFromRootBase(ctx, baseRepo, headUserID, maxForkTraverseLevel)
}
func findHeadRepoFromRootBase(ctx context.Context, baseRepo *repo_model.Repository, headUserID int64, traverseLevel int) (*repo_model.Repository, error) {
if traverseLevel == 0 {
return nil, nil
}
// test if we are lucky
repo, err := repo_model.GetUserFork(ctx, baseRepo.ID, headUserID)
if err != nil {
return nil, err
}
if repo != nil {
return repo, nil
}
firstLevelForkedRepos, err := repo_model.GetRepositoriesByForkID(ctx, baseRepo.ID)
if err != nil {
return nil, err
}
for _, repo := range firstLevelForkedRepos {
forked, err := findHeadRepoFromRootBase(ctx, repo, headUserID, traverseLevel-1)
if err != nil {
return nil, err
}
if forked != nil {
return forked, nil
}
}
return nil, nil
}

Some files were not shown because too many files have changed in this diff Show More