* Use Diesels MultiConnections Derive
With this PR we remove almost all custom macro's to create the multiple database type code. This is now handled by Diesel it self.
This removed the need of the following functions/macro's:
- `db_object!`
- `::to_db`
- `.from_db()`
It is also possible to just use one schema instead of multiple per type.
Also done:
- Refactored the SQLite backup function
- Some formatting of queries so every call is one a separate line, this looks a bit better
- Declare `conn` as mut inside each `db_run!` instead of having to declare it as `mut` in functions or calls
- Added an `ACTIVE_DB_TYPE` static which holds the currently active database type
- Removed `diesel_logger` crate and use Diesel's `set_default_instrumentation()`
If you want debug queries you can now simply change the log level of `vaultwarden::db::query_logger`
- Use PostgreSQL v17 in the Alpine images to match the Debian Trixie version
- Optimized the Workflows since `diesel_logger` isn't needed anymore
And on the extra plus-side, this lowers the compile-time and binary size too.
Signed-off-by: BlackDex <black.dex@gmail.com>
* Adjust query_logger and some other small items
Signed-off-by: BlackDex <black.dex@gmail.com>
* Remove macro, replaced with an function
Signed-off-by: BlackDex <black.dex@gmail.com>
* Implement custom connection manager
Signed-off-by: BlackDex <black.dex@gmail.com>
* Updated some crates to keep up2date
Signed-off-by: BlackDex <black.dex@gmail.com>
* Small adjustment
Signed-off-by: BlackDex <black.dex@gmail.com>
* crate updates
Signed-off-by: BlackDex <black.dex@gmail.com>
* Update crates
Signed-off-by: BlackDex <black.dex@gmail.com>
---------
Signed-off-by: BlackDex <black.dex@gmail.com>
* rename membership
rename UserOrganization to Membership to clarify the relation
and prevent confusion whether something refers to a member(ship) or user
* use newtype pattern
* implement custom derive macro IdFromParam
* add UuidFromParam macro for UUIDs
* add macros to Docker build
Co-authored-by: dfunkt <dfunkt@users.noreply.github.com>
---------
Co-authored-by: dfunkt <dfunkt@users.noreply.github.com>
- Refactored several code to use more modern syntax
- Made some checks a bit more strict
- Updated crates
Signed-off-by: BlackDex <black.dex@gmail.com>
* Change API inputs/outputs and structs to camelCase
* Fix fields and password history
* Use convert_json_key_lcase_first
* Make sends lowercase
* Update admin and templates
* Update org revoke
* Fix sends expecting size to be a string on mobile
* Convert two-factor providers to string
Improved sync speed by resolving the N+1 query issues.
Solves #1402 and Solves #1453
With this change there is just one query done to retreive all the
important data, and matching is done in-code/memory.
With a very large database the sync time went down about 3 times.
Also updated misc crates and Github Actions versions.
This is a rather large PR which updates the async branch to have all the
database methods as an async fn.
Some iter/map logic needed to be changed to a stream::iter().then(), but
besides that most changes were just adding async/await where needed.
When using MariaDB v10.5+ Foreign-Key errors were popping up because of
some changes in that version. To mitigate this on MariaDB and other
MySQL forks those errors are now catched, and instead of a replace_into
an update will happen. I have tested this as thorough as possible with
MariaDB 10.5, 10.4, 10.3 and the default MySQL on Ubuntu Focal. And
tested it again using sqlite, all seems to be ok on all tables.
resolves#1081. resolves#1065, resolves#1050
Diesel requires the following changes:
- Separate connection and pool types per connection, the generate_connections! macro generates an enum with a variant per db type
- Separate migrations and schemas, these were always imported as one type depending on db feature, now they are all imported under different module names
- Separate model objects per connection, the db_object! macro generates one object for each connection with the diesel macros, a generic object, and methods to convert between the connection-specific and the generic ones
- Separate connection queries, the db_run! macro allows writing only one that gets compiled for all databases or multiple ones
This includes migrations as well as Dockerfile's for amd64.
The biggest change is that replace_into isn't supported by Diesel for the
PostgreSQL backend, instead requiring the use of on_conflict. This
unfortunately requires a branch for save() on all of the models currently
using replace_into.
For now only folder notifications are sent (create, rename, delete).
The notifications are only tested between two web-vault sessions in different browsers, mobile apps and browser extensions are untested.
The websocket server is exposed in port 3012, while the rocket server is exposed in another port (8000 by default). To make notifications work, both should be accessible in the same port, which requires a reverse proxy.
My testing is done with Caddy server, and the following config:
```
localhost {
# The negotiation endpoint is also proxied to Rocket
proxy /notifications/hub/negotiate 0.0.0.0:8000 {
transparent
}
# Notifications redirected to the websockets server
proxy /notifications/hub 0.0.0.0:3012 {
websocket
}
# Proxy the Root directory to Rocket
proxy / 0.0.0.0:8000 {
transparent
}
}
```
This exposes the service in port 2015.
Mathijs van Veluw
Stefan Melmuk
Daniel García
Jake Howard
Michael Powers
janost
Miroslav Prasil
Daniel García