Resolving and Preventing Global Metadata Database Lock Issues in Riva On-Premise (2025 and Earlier)

Grace Wannapongsai
Grace Wannapongsai
  • Updated

Overview

Riva On-Premise 2025 or earlier may experience ProcessFailedReservations, database is locked, or disk I/O error errors when the Global Metadata database encounters locking or write-ahead log (WAL) issues.

These issues may occur when changes stored in the WAL file cannot be committed to the global.metadata SQLite database. Over time, the WAL file may grow significantly, potentially affecting synchronization operations.

This article outlines troubleshooting steps used by Riva Support to resolve oversized WAL files and provides maintenance recommendations to help reduce the likelihood of recurrence. We recommend ensuring that your Riva On-Premise environment is running the latest available version to benefit from database lock fixes and optimizations.

Applies to: Riva On-Premise 2025 and earlier.

Resolution

Step 1: Restart the Riva Sync Service (RSS)

First, attempt to resolve the issue by restarting RSS. This is the least invasive approach and may allow Riva to commit the WAL file automatically.

  1. Restart RSS.

  2. Monitor the Riva logs for the following messages:

    [RecycleDatabaseWalFile] RecycleDatabaseWalFile is starting...
[RecycleDatabaseWalFile] The Wal file was successfully recycled.
  3. Confirm that the WAL file has been recycled and that synchronization resumes successfully.

If the issue persists, proceed to Step 2.

Step 2: Manually Commit the WAL File Using SQLiteSpy

If restarting RSS does not resolve the issue, the WAL file may need to be committed manually.

Important: This procedure requires downtime and should be performed with assistance from Riva Support. Ensure a complete backup is available before modifying the database.

  1. Stop all Riva services, including the Gatherer, RSS, and all sync nodes.
  2. Copy the entire Global Metadata directory, including the database and associated WAL files, to a separate drive with sufficient available disk space.
  3. Create two copies of the directory:
    • Working copy: Used to perform the recovery.
    • Backup copy: Preserved in case the recovery is unsuccessful.
  4. Open the working copy of global.metadata in SQLiteSpy.
  5. Allow time for any WAL recovery or checkpoint activity to complete. Depending on the WAL file size, this may take a significant amount of time.

  6. If the WAL file is not committed automatically, execute the following command in SQLiteSpy:

    PRAGMA wal_checkpoint(TRUNCATE);

    Press F9 or select Execute from the menu bar.

  7. Verify that the checkpoint completed successfully before proceeding.
  8. Close the database connection and confirm that the recovered database is consistent and ready for restoration.
  9. Following Riva Support's validated restoration procedure, copy the recovered database back to the production directory.
  10. Start RSS and verify that it starts successfully.
  11. Start the Gatherer and the first sync node, then confirm that users begin syncing.
  12. Once synchronization is confirmed, start the remaining sync nodes in batches.

Warning: Do not manually delete the WAL file, as it may contain uncommitted transactions. Deleting it without a successful checkpoint may result in data loss.

Step 3: Run Database Maintenance (VACUUM)

After the WAL file has been successfully committed, the Global Metadata database will still contain unused space or fragmented data from repeated insert, update, and delete operations.

Perform GCM cleanup and run a VACUUM operation on the working copy of the Global Metadata database to reclaim unused space and reduce fragmentation.

  1. Perform the GCM cleanup on the working copy before running VACUUM.
  2. Open the working copy of global.metadata in SQLiteSpy.

  3. Execute the following SQL command:

    VACUUM;
  4. Allow the operation to complete without interruption.

Important Warnings:

  • At approximately 7 GB, the initial VACUUM operation could take 24+ hours.
  • The VACUUM operation requires at least twice the database size in available free disk space.
  • Run the GCM cleanup first, before performing the VACUUM operation.
  • Perform this procedure during a scheduled maintenance/blackout window.

Preventing Recurrence

The following maintenance recommendations may help reduce the likelihood of database locking issues in older Riva On-Premise environments.

1. Configure Blackout Times

Configure a daily blackout period, such as 03:00–04:00, during which synchronization activity can pause.

Blackout periods provide an opportunity for WAL file changes to be committed to the Global Metadata database without ongoing synchronization activity interfering with the operation.

Without an adequate interruption in database activity, WAL files may continue to grow.

2. Perform Regular Database Maintenance

Consider periodically running a VACUUM operation on the global.metadata SQLite database to reclaim unused space and reduce fragmentation.

This maintenance should be performed during a planned downtime window, with all relevant Riva services stopped and a verified backup available.

3. Restart RSS Regularly

If recurring disk I/O error or database is locked errors are observed, consider restarting the sync service daily rather than weekly.

4. Keep Riva Updated

Ensure you are running the latest available version of Riva On-Premise. Newer releases include database lock fixes and optimizations that can help prevent these issues from recurring.

5. Review Antivirus and Backup Exclusions

Antivirus and backup applications that access Riva's data or transaction directories may interfere with database operations or contribute to file locking.

Review the applicable Riva directory exclusions with your IT or security team to ensure these applications do not interfere with active database files. Any exclusions should follow your organization's security policies.