letro/letro-authentication-service
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

Remove the old migration tool and document the new one (#4447)

Browse Source
This commit is contained in:
Quentin Gliech
2025-04-30 13:52:31 +02:00
committed by GitHub
parent 5185dd2773 3e7b7d7463
commit 7b8ff852cc
44 changed files with 171 additions and 5584 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/SUMMARY.md
View File

@@ -12,7 +12,6 @@
- [Database setup](./setup/database.md)
- [Homeserver configuration](./setup/homeserver.md)
- [Configuring a reverse proxy](./setup/reverse-proxy.md)
- [Configuring .well-known](./setup/well-known.md)
- [Configure an upstream SSO provider](./setup/sso.md)
- [Running the service](./setup/running.md)
- [Migrating an existing homeserver](./setup/migration.md)
@@ -33,6 +32,7 @@
- [`database`](./reference/cli/database.md)
- [`manage`](./reference/cli/manage.md)
- [`server`](./reference/cli/server.md)
- [`syn2mas`](./reference/cli/syn2mas.md)
- [`worker`](./reference/cli/worker.md)
- [`templates`](./reference/cli/templates.md)
- [`doctor`](./reference/cli/doctor.md)

8
docs/reference/cli/config.md
View File

@@ -26,7 +26,7 @@ clients:
# ...
```
## `config generate`
## `config generate [--synapse-config <synapse-config>] [--output <output>]`
Generate a sample configuration file.
It generates random signing keys (`.secrets.keys`) and the cookie encryption secret (`.secrets.encryption`).
@@ -38,6 +38,10 @@ INFO generate:rsa: mas_config::oauth2: Done generating RSA key
INFO generate:ecdsa: mas_config::oauth2: Done generating ECDSA key
```
The `--synapse-config` option can be used to migrate over configuration options from an existing Synapse configuration.
The `--output` option can be used to specify the output file. If not specified, the output will be written to stdout.
## `config sync [--prune] [--dry-run]`
Synchronize the configuration with the database.
@@ -52,4 +56,4 @@ INFO cli.config.sync: Updating provider provider.id=01H3FDH2XZJS8ADKRGWM84PZTY
INFO cli.config.sync: Adding provider provider.id=01H3FDH2XZJS8ADKRGWM84PZTF
INFO cli.config.sync: Deleting client client.id=01GFWRB9MYE0QYK60NZP2YF905
INFO cli.config.sync: Updating client client.id=01GFWRB9MYE0QYK60NZP2YF904
```
```

29
docs/reference/cli/syn2mas.md Normal file
View File

@@ -0,0 +1,29 @@
# `syn2mas`
Tool to import data from an existing Synapse homeserver into MAS.
Global options:
- `--config <config>`: Path to the MAS configuration file.
- `--help`: Print help.
- `--synapse-config <synapse-config>`: Path to the Synapse configuration file.
- `--synapse-database-uri <synapse-database-uri>`: Override the Synapse database URI.
## `syn2mas check`
Check the setup for potential problems before running a migration
```console
$ mas-cli syn2mas check --config mas_config.yaml --synapse-config homeserver.yaml
```
## `syn2mas migrate [--dry-run]`
Migrate data from the homeserver to MAS.
The `--dry-run` option will perform a dry-run of the migration, which is safe to run without stopping Synapse.
It will perform a full data migration, but then empty the MAS database at the end to roll back.
```console
$ mas-cli syn2mas migrate --config mas_config.yaml --synapse-config homeserver.yaml
```

34
docs/setup/README.md
View File

@@ -11,43 +11,11 @@ The authentication service becomes the source of truth for user accounts and acc
At time of writing, the authentication service is meant to be run on a standalone domain name (e.g. `auth.example.com`), and the homeserver on another (e.g. `matrix.example.com`).
This domain will be user-facing as part of the authentication flow.
When a client initiates an authentication flow, it will discover the authentication service through the deployment `.well-known/matrix/client` endpoint.
This file will refer to an `issuer`, which is the canonical name of the authentication service instance.
Out of that issuer, it will discover the rest of the endpoints by calling the `[issuer]/.well-known/openid-configuration` endpoint.
By default, the `issuer` will match the root domain where the service is deployed (e.g. `https://auth.example.com/`), but it can be configured to be different.
An example setup could look like this:
- The deployment domain is `example.com`, so Matrix IDs look like `@user:example.com`
- The issuer chosen is `https://auth.example.com/`
- The homeserver is deployed on `matrix.example.com`
- The authentication service is deployed on `auth.example.com`
- Calling `https://example.com/.well-known/matrix/client` returns the following JSON:
```json
{
"m.homeserver": {
"base_url": "https://matrix.example.com"
},
"org.matrix.msc2965.authentication": {
"issuer": "https://auth.example.com/",
"account": "https://auth.example.com/account"
}
}
```
- Calling `https://auth.example.com/.well-known/openid-configuration` returns a JSON document similar to the following:
```json
{
"issuer": "https://auth.example.com/",
"authorization_endpoint": "https://auth.example.com/authorize",
"token_endpoint": "https://auth.example.com/oauth2/token",
"jwks_uri": "https://auth.example.com/oauth2/keys.json",
"registration_endpoint": "https://auth.example.com/oauth2/registration",
"//": "..."
}
```
- The homeserver is deployed on `matrix.example.com`
With the installation planned, it is time to go through the installation and configuration process.
The first section focuses on [installing the service](./installation.md).

175
docs/setup/migration.md
View File

@@ -1,51 +1,54 @@
# Migrating an existing homeserver
One of the design goals of MAS has been to allow it to be used to migrate an existing homeserver to an OIDC-based architecture.
One of the design goals of MAS has been to allow it to be used to migrate an existing homeserver, specifically without requiring users to re-authenticate and ensuring that all existing clients continue to work.
Specifically without requiring users to re-authenticate and that non-OIDC clients continue to work.
Features that are provided to support this include:
Features that support this include:
- Ability to import existing password hashes from Synapse
- Ability to import existing sessions and devices
- Ability to import existing access tokens linked to devices (ie not including short-lived admin puppeted access tokens)
- Ability to import existing access tokens
- Ability to import existing upstream IdP subject ID mappings
- Provides a compatibility layer for legacy Matrix authentication
There will be tools to help with the migration process itself. But these aren't quite ready yet.
## Preparing for the migration
The deployment is non-trivial so it is important to read through and understand the steps involved and make a plan before starting.
The deployment is non-trivial, so it is important to read through and understand the steps involved and make a plan before starting.
### Get `syn2mas`
### Is your setup ready to be migrated?
The easiest way to get `syn2mas` is through [`npm`](https://www.npmjs.com/package/@vector-im/syn2mas):
#### SAML2 and LDAP Single Sign-On Providers are not supported
```sh
npm install -g @vector-im/syn2mas
```
A deployment that requires SAML or LDAP-based authentication should use a service like [Dex](https://github.com/dexidp/dex) to bridge between the SAML provider and the authentication service.
MAS differs from Synapse in that it does **not** have built-in support for SAML or LDAP-based providers.
### Run the migration advisor
#### Custom password providers are not supported
You can use the advisor mode of the `syn2mas` tool to identify extra configuration steps or issues with the configuration of the homeserver.
If your Synapse homeserver currently uses a custom password provider module, please note that MAS does not support these.
```sh
syn2mas --command=advisor --synapseConfigFile=homeserver.yaml
```
#### SQLite databases are not supported
This will output `WARN` entries for any identified actions and `ERROR` entries in the case of any issues that will prevent the migration from working.
It is worth noting that MAS currently only supports PostgreSQL as a database backend.
The migration tool only supports reading from PostgreSQL for the Synapse database as well.
### Install and configure MAS alongside your existing homeserver
Follow the instructions in the [installation guide](installation.md) to install MAS alongside your existing homeserver.
You'll need a blank PostgreSQL database for MAS to use; it does not share the database with the homeserver.
MAS provides a tool to generate a configuration file based on your existing Synapse configuration. This is useful for kickstarting your new configuration.
```sh
mas-cli config generate --synapse-config homeserver.yaml --output mas_config.yaml
```
When using this tool, be careful to examine the log output for any warnings about unsupported configuration options.
#### Local passwords
Synapse uses bcrypt as its password hashing scheme while MAS defaults to using the newer argon2id.
Synapse uses bcrypt as its password hashing scheme, while MAS defaults to using the newer argon2id.
You will have to configure the version 1 scheme as bcrypt for migrated passwords to work.
It is also recommended that you keep argon2id as version 2 so that once users log in, their hashes will be updated to the newer recommended scheme.
If you have a `pepper` set in the `password_config` section of your Synapse config, then you need to specify this `pepper` as the `secret` field for your `bcrypt` scheme.
It is also recommended that you keep argon2id as version 2 so that once users log in, their hashes will be updated to the newer, recommended scheme.
Example passwords configuration:
```yml
@@ -60,57 +63,143 @@ passwords:
algorithm: argon2id
```
If you have a pepper configured in your Synapse password configuration, you'll need to match that on version 1 of the equivalent MAS configuration.
The migration checker will inform you if this has not been configured properly.
### Map any upstream SSO providers
If you are using an upstream SSO provider then you will need to provision the upstream provide in MAS manually.
If you are using an upstream SSO provider, then you will need to configure the upstream provider in MAS manually.
Each upstream provider will need to be given as an `--upstreamProviderMapping` command line option to the import tool.
MAS does not support SAML or LDAP upstream providers.
If you are using one of these, you will need to use an adapter such as Dex at this time, but we have not yet documented this procedure.
### Prepare the MAS database
Each upstream provider that was used by at least one user in Synapse will need to be configured in MAS.
Once the database is created, it still needs to have its schema created and synced with the configuration.
This can be done with the following command:
Set the `synapse_idp_id` attribute on the provider to:
```sh
mas-cli config sync
- `"oidc"` if you used an OIDC provider in Synapse's legacy `oidc_config` configuration section.
- `"oidc-myprovider"` if you used an OIDC provider in Synapse's `oidc_providers` configuration list, with a `provider` of `"myprovider"`.
(This is because Synapse prefixes the provider ID with `oidc-` internally.)
Without the `synapse_idp_id`s being set, `mas-cli syn2mas` does not understand which providers in Synapse correspond to which provider in MAS.
For example, if your Synapse configuration looked like this:
```yaml
oidc_providers:
- idp_id: dex
idp_name: "My Dex server"
issuer: "https://example.com/dex"
client_id: "synapse"
client_secret: "supersecret"
scopes: ["openid", "profile", "email"]
user_mapping_provider:
config:
localpart_template: "{{ user.email.split('@')[0].lower() }}"
email_template: "{{ user.email }}"
display_name_template: "{{ user.name|capitalize }}"
```
### Do a dry-run of the import to test
Then the equivalent configuration in MAS would look like this:
```sh
syn2mas --command migrate --synapseConfigFile homeserver.yaml --masConfigFile config.yaml --dryRun
```yaml
upstream_oauth2:
providers:
- id: 01JSHPZHAXC50QBKH67MH33TNF
synapse_idp_id: oidc-dex
issuer: "https://example.com/dex"
human_name: "My Dex server"
client_id: "synapse"
client_secret: "supersecret"
token_endpoint_auth_method: client_secret_basic
scope: "email openid profile"
claims_imports:
localpart:
action: require
template: "{{ user.email.split('@')[0].lower() }}"
displayname:
action: force
template: "{{ user.name|capitalize }}"
email:
action: force
template: "{{ user.email }}"
```
If no errors are reported then you can proceed to the next step.
The migration checker will inform you if a provider is missing from MAS' config.
### Run the migration checker
You can use the `check` command of the `syn2mas` tool to identify configuration problems before starting the migration.
You do not need to stop Synapse to run this command.
```sh
mas-cli syn2mas check --config mas_config.yaml --synapse-config homeserver.yaml
```
This may output a list of errors and warnings.
If you have any errors, you must resolve them before starting the migration.
If you have any warnings, please read and understand them, and possibly resolve them.
Resolving warnings is not strictly required before starting the migration.
### Run the migration in test mode (dry-run)
MAS can perform a dry-run of the import, which is safe to run without stopping Synapse.
It will perform a full data migration but then empty the MAS database at the end to roll back.
This means it is safe to run multiple times without worrying about resetting the MAS database.
It also means the time this dry-run takes is representative of the time it will take to perform the actual migration.
```sh
mas-cli syn2mas migrate --config mas_config.yaml --synapse-config homeserver.yaml --dry-run
```
## Doing the migration
Having done the preparation, you can now proceed with the actual migration. Note that this will require downtime for the homeserver and is not easily reversible.
Having completed the preparation, you can now proceed with the actual migration. Note that this will require downtime for the homeserver and is not easily reversible.
### Backup your data
### Backup your data and configuration
As with any migration, it is important to backup your data before proceeding.
As with any migration, it is important to back up your data before proceeding.
### Shutdown the homeserver
We also suggest making a backup copy of your homeserver's known good configuration before making any changes to enable MAS integration.
This is to ensure that no new sessions are created whilst the migration is in progress.
### Shut down the homeserver
### Configure the homeserver
This ensures that no new sessions are created while the migration is in progress.
### Configure the homeserver to enable MAS integration
Follow the instructions in the [homeserver configuration guide](homeserver.md) to configure the homeserver to use MAS.
### Do the import
Run `syn2mas` in non-dry-run mode.
Once the homeserver has been stopped, MAS has been configured (but is not running!), and you have a successful migration check, run `syn2mas`'s `migrate` command.
```sh
syn2mas --command migrate --synapseConfigFile homeserver.yaml --masConfigFile config.yaml --dryRun false
mas-cli syn2mas migrate --config mas_config.yaml --synapse-config homeserver.yaml
```
#### What to do if it goes wrong
If the migration fails with an error:
- You can try to fix the error and make another attempt by re-running the command; or
- You can revert your homeserver configuration (so MAS integration is disabled once more) and abort the migration for now. In this case, you should not start MAS up.
In *some cases*, MAS may have written to its own database during a failed migration, causing it to complain in subsequent runs.
In this case, you can safely delete and recreate the MAS database, then start over.
In *any case*, the migration tool itself **will not** write to the Synapse database, so as long as MAS hasn't been started, it is safe to roll back the migration without restoring the Synapse database.
Please report migration failures to the developers.
### Start up the homeserver
Start up the homeserver again with the new configuration.
### Update or serve the .well-known
### Start up MAS
The `.well-known/matrix/client` needs to be served as described [here](./well-known.md).
Now you can start MAS.

23
docs/setup/well-known.md
View File

@@ -1,23 +0,0 @@
# .well-known configuration
A `.well-known/matrix/client` file is required to be served to allow clients to discover the authentication service.
If no `.well-known/matrix/client` file is served currently then this will need to be enabled.
If the homeserver is Synapse and serving this file already then the correct values will already be included when the homeserver is [configured to use MAS](./homeserver.md).
If the .well-known is hosted elsewhere then `org.matrix.msc2965.authentication` entries need to be included similar to the following:
```json
{
"m.homeserver": {
"base_url": "https://matrix.example.com"
},
"org.matrix.msc2965.authentication": {
"issuer": "https://example.com/",
"account": "https://auth.example.com/account"
}
}
```
For more context on what the correct values are, see [here](./).