110 Commits

Author SHA1 Message Date
Quentin Gliech
d7bd50f12b Track skipped entities in the syn2mas progress 2025-03-14 13:29:19 +00:00
Olivier 'reivilibre
35ef586447 fixup! syn2mas: Add progress reporting to log and to opentelemetry metrics
Add metrics directly within syn2mas, no background thread
2025-03-13 14:50:32 +00:00
Olivier 'reivilibre
3a29eef432 syn2mas: Add progress reporting to log and to opentelemetry metrics 2025-03-12 18:49:36 +00:00
Olivier 'reivilibre
b670d1dc19 Add dependency on arc-swap to syn2mas, moving it to a workspace dependency 2025-03-12 14:55:03 +00:00
Olivier 'reivilibre
75ca08b7f9 Remove TODO on MIGRATOR 2025-03-12 14:38:32 +00:00
Olivier 'reivilibre
163b1e8d79 syn2mas: improve performance of reading access tokens / refresh tokens 2025-03-12 14:38:32 +00:00
Quentin Gliech
51e1c11df9 Order the OAuth providers in the UI by their order in the config file (#4199) 2025-03-12 12:12:39 +01:00
reivilibre
8553efdab4 syn2mas: better performance, output tweaks, tracing tweaks, access token fixes (#4175) 2025-03-12 10:56:56 +00:00
Quentin Gliech
c4f7cad172 Update test snapshots 2025-03-12 11:47:07 +01:00
Quentin Gliech
bf496e8d81 syn2mas: import the deactivation and locked status independently 2025-03-11 17:35:13 +01:00
Quentin Gliech
d2f3b56cd0 Better report errors when a writing task fails 2025-03-10 13:53:36 +00:00
Quentin Gliech
8ee4e68d6d Read and write users in different tasks 2025-03-10 13:53:36 +00:00
Quentin Gliech
c45500be72 Use rustc' FxHasher instead of the std one 2025-03-10 13:36:12 +00:00
Quentin Gliech
0b9036601a Better tracing during the migration 2025-03-10 13:36:12 +00:00
Quentin Gliech
32071e1833 Make sure tracing spans are propagated in tokio::task::spawn 2025-03-10 13:36:11 +00:00
Olivier 'reivilibre
07dd9cd9ec Log timings of migrations and index/constraint rebuilds 2025-03-10 13:25:59 +00:00
Olivier 'reivilibre
8c1a8dd8a1 Add spans for the constraint/index drop operations 2025-03-10 13:23:59 +00:00
Olivier 'reivilibre
6fdfb263b5 Don't include access tokens where the referenced device has vanished 2025-03-10 13:23:59 +00:00
Olivier 'reivilibre
eff33289a5 Don't read refresh tokens that don't have associated access tokens
These appear to be accidental and should not be migrated.
2025-03-10 13:23:59 +00:00
Olivier 'reivilibre
4f4fe822d9 For performance, switch to a row count estimate for users and devices 2025-03-10 13:23:59 +00:00
Olivier 'reivilibre
b0c958007f Replace panic on unfinished buffers with an error
This doesn't get clobbered by the progress bar
2025-03-10 13:23:55 +00:00
Olivier 'reivilibre
bfcd0f8209 Downgrade presence of guests and unsupported 3pids to warnings
Since migration is supported, but the data won't be used
2025-03-10 13:20:46 +00:00
Olivier 'reivilibre
155b4f31d2 Remove obsolete OIDC provider warning
We now produce an error for unmapped OIDC providers!
2025-03-10 13:20:46 +00:00
dependabot[bot]
895678f968 build(deps): bump uuid from 1.14.0 to 1.15.1
Bumps [uuid](https://github.com/uuid-rs/uuid) from 1.14.0 to 1.15.1.
- [Release notes](https://github.com/uuid-rs/uuid/releases)
- [Commits](https://github.com/uuid-rs/uuid/compare/v1.14.0...v1.15.1)

---
updated-dependencies:
- dependency-name: uuid
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-07 14:05:02 +00:00
Quentin Gliech
6e65db5aca Handle AS users with invalid localparts edge case 2025-03-04 12:38:51 +01:00
Quentin Gliech
9dc1b39fa1 Properly ignore devices, threepids and access tokens from AS users (#4122) 2025-03-04 12:38:14 +01:00
Quentin Gliech
004119dbce Pass the MasWriter as owned to the various migration functions (#4120) 2025-03-04 12:37:23 +01:00
Quentin Gliech
fa38879558 syn2mas: use NonNilUuid for MAS user IDs 2025-03-04 11:55:59 +01:00
Quentin Gliech
435911bb85 Revert "Ignore colons in localparts when importing users"
This reverts commit bd0b04ae4d.
2025-03-03 13:52:25 +01:00
Quentin Gliech
8e7322f0e8 Actualy read appservice users, but don't insert them 2025-02-27 14:11:42 +01:00
Quentin Gliech
500f3530e3 Ignore 'guest_devices' in synapse 2025-02-27 14:07:23 +01:00
Quentin Gliech
2d90e25077 Ignore '-' as ip on devices 2025-02-27 14:07:23 +01:00
Quentin Gliech
bd0b04ae4d Ignore colons in localparts when importing users 2025-02-27 14:07:22 +01:00
Quentin Gliech
347e37ad28 Pass the MasWriter as owned to the various migration functions 2025-02-27 09:51:56 +01:00
Quentin Gliech
10d7338934 Bump most Rust dependencies 2025-02-24 12:12:07 +01:00
Quentin Gliech
56d9c7e63b Upgrade to Rust 1.85 and edition 2024 2025-02-21 16:15:02 +01:00
dependabot[bot]
686b05b04f build(deps): bump uuid from 1.13.2 to 1.14.0
Bumps [uuid](https://github.com/uuid-rs/uuid) from 1.13.2 to 1.14.0.
- [Release notes](https://github.com/uuid-rs/uuid/releases)
- [Commits](https://github.com/uuid-rs/uuid/compare/v1.13.2...v1.14.0)

---
updated-dependencies:
- dependency-name: uuid
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-02-21 13:53:19 +00:00
dependabot[bot]
e2f7c513be build(deps): bump uuid from 1.13.1 to 1.13.2
Bumps [uuid](https://github.com/uuid-rs/uuid) from 1.13.1 to 1.13.2.
- [Release notes](https://github.com/uuid-rs/uuid/releases)
- [Commits](https://github.com/uuid-rs/uuid/compare/1.13.1...v1.13.2)

---
updated-dependencies:
- dependency-name: uuid
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-02-19 16:53:29 +00:00
Quentin Gliech
ad48e89aa6 Pass the migration state as owned to the various stages 2025-02-10 10:14:20 +01:00
Quentin Gliech
c6fdd82432 Encapsulate migration state in a single structure 2025-02-07 15:02:30 +01:00
Quentin Gliech
d1d73af3ce Remove the lifetime parameter from MasWriter 2025-02-07 12:57:21 +01:00
Quentin Gliech
260834bc01 Make the MAS connection owned in the locked database struct 2025-02-07 12:53:51 +01:00
dependabot[bot]
8b229eddaf build(deps): bump uuid from 1.12.1 to 1.13.1
Bumps [uuid](https://github.com/uuid-rs/uuid) from 1.12.1 to 1.13.1.
- [Release notes](https://github.com/uuid-rs/uuid/releases)
- [Commits](https://github.com/uuid-rs/uuid/compare/1.12.1...1.13.1)

---
updated-dependencies:
- dependency-name: uuid
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-02-05 13:31:06 +00:00
Olivier 'reivilibre
dacfa9045d Don't return errors when finding rows dependent upon appservice users 2025-02-05 11:42:04 +01:00
Olivier 'reivilibre
ac58b4f326 Support reading and writing guests 2025-02-05 11:41:52 +01:00
Olivier 'reivilibre
dc305ddc40 Split access tokens between refreshable and unrefreshable ones 2025-02-05 11:41:45 +01:00
Olivier 'reivilibre
73ccecf8e8 Add MasWriter support for compat refresh tokens + some migration progress 2025-02-05 11:38:50 +01:00
Olivier 'reivilibre
900413cc43 Add MasWriter support for compat access tokens 2025-02-05 11:38:06 +01:00
Olivier 'reivilibre
0cd8106624 Add MasWriter support for compat sessions 2025-02-05 11:37:59 +01:00
Olivier 'reivilibre
719681f3e7 Add SynapseReader support for devices 2025-02-05 11:36:46 +01:00