Commit Graph

56 Commits

Author SHA1 Message Date
bircni
240d0efa7e perf: extend action c_u index to include created_unix for faster dashboard feeds (#38076)
Adds `created_unix` as the third column of the `c_u` composite index on
the `action` table, changing it from `(user_id, is_deleted)` to
`(user_id, is_deleted, created_unix)`.

Migration 337 drops and recreates the index. No data is touched.

## Root causes

#32333 introduced the `c_u` index to speed up dashboard queries, but
defined it as `(user_id, is_deleted)` — without `created_unix`.

#3368 The simple query is now efficient enough for the database to
actually use `c_u`, but because `created_unix` is absent from the index,
the database must load and sort **every** matching row before returning
the first page of 20.

The existing `c_u_d` index `(created_unix, user_id, is_deleted)` does
not help because its leading column is `created_unix`, which can't be
used for an equality seek on `user_id`.

Those two caused this issue:
https://github.com/go-gitea/gitea/issues/38075

With the fix, the database seeks directly to `(user_id=X,
is_deleted=false)` and walks `created_unix` in descending order,
stopping after 20 rows.

Fixes https://github.com/go-gitea/gitea/issues/38075
2026-06-17 23:37:55 +03:00
Lunny Xiao
61b1a39efe chore: Move import path from code.gitea.io/gitea to gitea.dev (#37873) 2026-05-26 15:49:31 -07:00
Lunny Xiao
f2a1271f16 fix: Unify public-only token filtering in API queries and repo access checks (#37118)
This PR closes remaining `public-only` token gaps in the API by making
the restriction apply consistently across repository, organization,
activity, notification, and authenticated `/api/v1/user/...` routes.

Previously, `public-only` tokens were still able to:
- receive private results from some list/search/self endpoints,
- access repository data through ID-based lookups,
- and reach several authenticated self routes that should remain
unavailable for public-only access.

This change treats `public-only` as a cross-cutting visibility boundary:
- list/search endpoints now filter private resources consistently,
- repository lookups enforce the same restriction even when addressed
indirectly,
- and self routes that inherently expose or mutate private account state
now reject `public-only` tokens.

---
Generated by a coding agent with Codex 5.2

---------

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Claude (Opus 4.7) <noreply@anthropic.com>
2026-05-18 11:36:42 -07:00
wxiaoguang
068b59aa97 Make GetPossibleUserByID can handle deleted user (#37430)
Make sure deleted user won't cause 500 error, simplify the caller's code
2026-04-26 16:57:53 +00:00
wxiaoguang
e837c998b7 Fix context usages (#35348) 2025-08-27 11:00:01 +00:00
silverwind
1f35435b81 Run gopls modernize on codebase (#34751)
Recent modernize fixes:
https://github.com/golang/tools/commits/master/gopls/internal/analysis/modernize
2025-06-18 01:48:09 +00:00
Lunny Xiao
a14db5c5e3 Fix ghost user in feeds when pushing in an actions, it should be gitea-actions (#34703) 2025-06-17 23:44:35 +00:00
ChristopherHX
a0595add72 Fix remove org user failure on mssql (#34449)
* mssql does not support fetching 0 repositories
  * remove paging by NumRepos that might be 0
* extend admin api test to purge user 2

Fixes #34448

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-05-14 02:33:56 +08:00
Lunny Xiao
a4df01b580 Optimize total count of feed when loading activities in user dashboard. (#33841)
Two SQLs are very slow when `action` table have over 5M records.

```
database duration=1.8881s db.sql="SELECT created_unix DIV 900 * 900 AS timestamp, count(user_id) as contributions FROM `action` WHERE user_id=? AND act_user_id=? AND (created_unix > ?) GROUP BY timestamp ORDER BY timestamp"

database duration=1.5408s db.sql="SELECT count(*) FROM `action` WHERE (user_id = ?) AND (is_deleted = ?)"
```

This will cache the count for the first loading or when the activities
changed.
2025-03-20 10:46:18 -07:00
Lunny Xiao
4a7ab0abf0 Optimize heatmap query (#33853)
When there are over 5M records on `action` table, the heatmap on
dashboard is very slow as below SQL.
```
database duration=1.8881s db.sql="SELECT created_unix DIV 900 * 900 AS timestamp, count(user_id) as contributions FROM `action` WHERE user_id=? AND act_user_id=? AND (created_unix > ?) GROUP BY timestamp ORDER BY timestamp"
```

This PR add a new index for `action` table with columns `user_id`,
`act_user_id` and `created_unix` so that this query will become about 6
times faster than before.
2025-03-20 09:30:45 -07:00
Lunny Xiao
ae63568ce3 Move notifywatch to service layer (#33825)
No logic change.
2025-03-10 00:54:25 +00:00
TheFox0x7
4c4c56c7cd Decouple context from repository related structs (#33823)
Calls that required context implicitly are made to pass it as argument
2025-03-08 13:12:46 -08:00
Lunny Xiao
76b7f95a27 Optimize user dashboard loading (#33686)
Fix #33582
Fix #31698

When a user login, the dashboard should load all feed belongs to him
with no any conditions. The complicated conditions should be applied
only for another user view this user's profile.
2025-02-24 11:29:32 -08:00
Lunny Xiao
3e7ec826d3 Add missed auto merge feed message on dashboard (#33309) 2025-01-26 18:33:05 +00:00
wxiaoguang
2ea929a952 Refactor RefName (#33234)
And fix some FIXMEs
2025-01-13 14:01:53 +08:00
wxiaoguang
9bfa9f450d Refactor "string truncate" (#32984) 2024-12-26 11:56:03 +08:00
Lunny Xiao
e4c4629465 Move RepoTransfer from models to models/repo sub package (#32506)
`RepoTransfer` now is at models, but if we want to move it into `repo`
model, it will depend on `Team`. So this PR also makes repo model depend
on org model to make it possible. Just refactor, no code change.

- [x] Move `DeleteOrganization` from `models/organization` to service
layer
- [x] Move `AccessibleTeamReposEnv` to `models/repo`
- [x] Move `RepoTransfer` from `models` to `models/repo`
- [x] Merge `getUserTeamIDs` and `GetUserTeamIDs`, Merge `GetUserTeams`
and `getUserTeams`.
- [x] Remove `Team`'s `Repos []*repo_model.Repository` to avoid dependency recycle.
2024-12-18 11:44:16 +08:00
Lunny Xiao
1ed5f379b9 Move GetFeeds to service layer (#32526)
Move GetFeeds from models to service layer, no code change.
2024-11-29 17:53:49 +00:00
wxiaoguang
44909f6e2c Fix markup render regression and fix some tests (#32640)
Fix #32639,
https://github.com/go-gitea/gitea/issues/32608#issuecomment-2497918210

By the way, fix some incorrect SQLs (use single quote but not double
quote)
2024-11-26 03:04:55 +08:00
wxiaoguang
633785a5f3 Refactor markup render system (#32612)
This PR removes (almost) all path tricks, and introduces "renderhelper"
package.

Now we can clearly see the rendering behaviors for comment/file/wiki,
more details are in "renderhelper" tests.

Fix #31411 , fix #18592, fix #25632 and maybe more problems. (ps: fix
#32608 by the way)
2024-11-24 16:18:57 +08:00
wxiaoguang
985e2a8af3 Fix nil panic if repo doesn't exist (#32501)
fix  #32496
2024-11-14 12:17:58 +08:00
Lunny Xiao
913be9e8ac Add new index for action to resolve the performance problem (#32333)
Fix #32224
2024-11-06 22:04:48 +00:00
Lunny Xiao
3f9e360142 Don't join repository when loading action table data (#32127) 2024-09-26 04:50:30 +00:00
Lunny Xiao
85b1f3080c Improve get feed with pagination (#31821)
Fix #31752

@somera

---------

Co-authored-by: delvh <dev.lh@web.de>
2024-09-02 19:05:09 +00:00
wxiaoguang
f4f4e18b14 Filter out duplicate action(activity) items for a repository (#30957)
Fix #20986
2024-05-14 13:47:03 +00:00
Lunny Xiao
aed3b53abd Some performance optimization on dashboard and issues page (#29010)
This PR do some loading speed optimization for feeds user interface
pages.
- Load action users batchly but not one by one.
- Load action repositories batchly but not one by one.
- Load action's Repo Owners batchly but not one by one.
- Load action's possible issues batchly but not one by one.
- Load action's possible comments batchly but not one by one.
2024-03-12 04:57:19 +00:00
wxiaoguang
1d2548949a Avoid issue info panic (#29625)
Fix #29624
2024-03-06 13:12:44 +00:00
wxiaoguang
3f081d4b54 Rename Action.GetDisplayName to GetActDisplayName (#29540)
To avoid conflicting with User.GetDisplayName, because there is no data
type in template.

And it matches other methods like GetActFullName / GetActUserName
2024-03-02 15:30:18 +00:00
Lunny Xiao
3a66762130 Revert "Speed up loading the dashboard on mysql/mariadb (#28546)" (#29006)
This reverts commit fa8c3beb26. #28546 
Because it seems performance become worse.
2024-02-01 14:25:37 +08:00
Lunny Xiao
fa8c3beb26 Speed up loading the dashboard on mysql/mariadb (#28546)
Fixes #28155
2024-01-14 15:34:25 +01:00
silverwind
ce83609ff6 Upgrade to golangci-lint@v1.55.0 (#27756)
https://github.com/golangci/golangci-lint/releases/tag/v1.55.0
2023-10-24 02:54:59 +00:00
JakobDev
4636f56e7b Add Index to action.user_id (#27403)
Another Column that needs a Index. Found at
https://codeberg.org/forgejo/discussions/issues/61#issuecomment-1258744.

Co-authored-by: Giteabot <teabot@gitea.io>
2023-10-03 21:41:25 -04:00
JakobDev
cc5df26680 Even more db.DefaultContext refactor (#27352)
Part of #27065

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: delvh <dev.lh@web.de>
2023-10-03 10:30:41 +00:00
JakobDev
cf0df023be More db.DefaultContext refactor (#27265)
Part of #27065

This PR touches functions used in templates. As templates are not static
typed, errors are harder to find, but I hope I catch it all. I think
some tests from other persons do not hurt.
2023-09-29 12:12:54 +00:00
CaiCandong
30cea70dd8 Refactor dashboard/feed.tmpl (#26956)
- ~~Refactor `ActionType` to `models/activities/action_type.go`~~
- Replace the magic number in `feed.tmlp` with `InAction`
2023-09-07 16:23:13 +02:00
wxiaoguang
e4b1ea6f15 Bypass MariaDB performance bug of the "IN" sub-query, fix incorrect IssueIndex (#26279)
Close #26277
Fix #26285
2023-08-07 12:23:59 +02:00
Lunny Xiao
b167f35113 Add context parameter to some database functions (#26055)
To avoid deadlock problem, almost database related functions should be
have ctx as the first parameter.
This PR do a refactor for some of these functions.
2023-07-22 22:14:27 +08:00
Lunny Xiao
f9cfd6ce5b Use the type RefName for all the needed places and fix pull mirror sync bugs (#24634)
This PR replaces all string refName as a type `git.RefName` to make the
code more maintainable.

Fix #15367
Replaces #23070 
It also fixed a bug that tags are not sync because `git remote --prune
origin` will not remove local tags if remote removed.

We in fact should use `git fetch --prune --tags origin` but not `git
remote update origin` to do the sync.

Some answer from ChatGPT as ref.

> If the git fetch --prune --tags command is not working as expected,
there could be a few reasons why. Here are a few things to check:
> 
>Make sure that you have the latest version of Git installed on your
system. You can check the version by running git --version in your
terminal. If you have an outdated version, try updating Git and see if
that resolves the issue.
> 
>Check that your Git repository is properly configured to track the
remote repository's tags. You can check this by running git config
--get-all remote.origin.fetch and verifying that it includes
+refs/tags/*:refs/tags/*. If it does not, you can add it by running git
config --add remote.origin.fetch "+refs/tags/*:refs/tags/*".
> 
>Verify that the tags you are trying to prune actually exist on the
remote repository. You can do this by running git ls-remote --tags
origin to list all the tags on the remote repository.
> 
>Check if any local tags have been created that match the names of tags
on the remote repository. If so, these local tags may be preventing the
git fetch --prune --tags command from working properly. You can delete
local tags using the git tag -d command.

---------

Co-authored-by: delvh <dev.lh@web.de>
2023-05-26 01:04:48 +00:00
Zettat123
29637b03b2 Fix commits pushed with deploy keys not shown in dashboard (#24521)
Fix #21324

In the current logic, if the `Actor` user is not an admin user, all
activities from private organizations won't be shown even if the `Actor`
user is a member of the organization.

As mentioned in the issue, when using deploy key to make a commit and
push, the activity's `act_user_id` will be the id of the organization so
the activity won't be shown to non-admin users because the visibility of
the organization is private.

55a5717760/models/activities/action.go (L490-L503)

This PR improves this logic so the activities of private organizations
can be shown.
2023-05-10 04:14:58 +00:00
Zettat123
6b0df6d8da Add activity feeds API (#23494)
Close #5666

Add APIs for getting activity feeds.
2023-04-04 21:35:31 +08:00
Zettat123
b958dba1a0 Improve indices for action table (#23532)
Close #21611 

Add the index mentioned in
https://github.com/go-gitea/gitea/issues/21611#issuecomment-1451113252 .
Since we already have an index for `("created_unix", "user_id",
"is_deleted")` columns on PostgreSQL, I removed the database type check
to apply this index to all types of databases.
2023-03-24 23:44:33 +08:00
Jason Song
c84238800b Refactor setting.Database.UseXXX to methods (#23354)
Replace #23350.

Refactor `setting.Database.UseMySQL` to
`setting.Database.Type.IsMySQL()`.

To avoid mismatching between `Type` and `UseXXX`.

This refactor can fix the bug mentioned in #23350, so it should be
backported.
2023-03-07 18:51:06 +08:00
Brecht Van Lommel
f4920c9c7f Add pagination for dashboard and user activity feeds (#22937)
Previously only the last few activities where available. This works for
all activity and for activity on a date chosen on the heatmap.
2023-02-24 16:15:10 -05:00
yp05327
bd66fa586a Rename repo.GetOwner to repo.LoadOwner (#22967)
Fixes https://github.com/go-gitea/gitea/issues/22963

---------

Co-authored-by: Yarden Shoham <hrsi88@gmail.com>
2023-02-18 20:11:03 +08:00
Lunny Xiao
769be877f2 Use link in UI which returned a relative url but not html_url which contains an absolute url (#21986)
partially fix #19345

This PR add some `Link` methods for different objects. The `Link`
methods are not different from `HTMLURL`, they are lack of the absolute
URL. And most of UI `HTMLURL` have been replaced to `Link` so that users
can visit them from a different domain or IP.

This PR also introduces a new javascript configuration
`window.config.reqAppUrl` which is different from `appUrl` which is
still an absolute url but the domain has been replaced to the current
requested domain.
2023-02-06 12:09:18 -06:00
Jason Song
6cf09ccab4 Use complete SHA to create and query commit status (#22244)
Fix #13485.

Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-12-27 21:12:49 +08:00
Lunny Xiao
0a7d3ff786 refactor some functions to support ctx as first parameter (#21878)
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: Lauris BH <lauris@nix.lv>
2022-12-03 10:48:26 +08:00
flynnnnnnnnnn
e81ccc406b Implement FSFE REUSE for golang files (#21840)
Change all license headers to comply with REUSE specification.

Fix #16132

Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
2022-11-27 18:20:29 +00:00
KN4CK3R
044c754ea5 Add context.Context to more methods (#21546)
This PR adds a context parameter to a bunch of methods. Some helper
`xxxCtx()` methods got replaced with the normal name now.

Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-11-19 16:12:33 +08:00
Lunny Xiao
34283a74e8 Allow detect whether it's in a database transaction for a context.Context (#21756)
Fix #19513

This PR introduce a new db method `InTransaction(context.Context)`,
and also builtin check on `db.TxContext` and `db.WithTx`.
There is also a new method `db.AutoTx` has been introduced but could be used by other PRs.

`WithTx` will always open a new transaction, if a transaction exist in context, return an error.
`AutoTx` will try to open a new transaction if no transaction exist in context.
That means it will always enter a transaction if there is no error.

Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: 6543 <6543@obermui.de>
2022-11-12 21:18:50 +01:00