gogstea

Migration from Gogs to Gitea with Less Pain

This project provides scripts and utilities to migrate data from Gogs (MySQL) to Gitea (PostgreSQL) with minimal disruption. The overall approach is to extract data from the Gogs database into CSV files and then import that data into Gitea via its API.

There are two main parts to the migration process:

1. Restoring a Gogs Database Dump in a Container
2. Generating CSV Files from the Gogs Database

High Level Operation

1. Export Data from Gogs:
   Run SQL queries against the Gogs MySQL database to extract data (users, organizations, collaborators, etc.) and save the results in CSV format.

2. Import into Gitea:
   Use the CSV files as input for a Go script that reads each CSV file and then uses the Gitea API (typically under the /admin endpoint) to create the corresponding objects in the Gitea PostgreSQL database.

For example, the workflow for migrating users is as follows:

source := gogs.mysql // SELECT * FROM user WHERE email != ''; --> export to CSV
dst := gitea.postgres // Gitea API /admin --> Go script reads the CSV and posts data to Gitea.
copy(dst, source)

Part 1: Restoring a Gogs Database Dump

Before running any migration scripts, you need to have a dump of your Gogs MySQL database. You can obtain this dump using your preferred MySQL backup tool. Once you have the dump file, use the provided shell script (init-gogs-db.sh) to restore the dump in a Docker container.

How to Use the init-gogs-db.sh Script

1. Prepare the Dump File:
   Ensure you have a MySQL dump file from your Gogs installation.

2. Run the Script:
   Execute the script and pass the path to your dump file as an argument. For example:

   ./init-gogs-db.sh /path/to/gogsdump.sql
   

The script will perform the following steps:

• Stop and remove any existing container named gogs-db.
• Run a new MySQL container with the hostname gogs-db and a preset root password.
• Copy your dump file into the container.
• Execute the database restore command inside the container.

Part 2: Generating CSV Files from the Gogs Database

After your database is restored and accessible, you can generate CSV files for various entities (users, organizations, collaborators, etc.) using the provided Go script. This script connects to the Gogs MySQL database, executes SQL queries, and writes the results into CSV files.

How to Generate the CSV Files

1- Ensure Your Database is Restored: Follow Part 1 to restore your Gogs database in a container.

2- Configure the Script: The script reads configuration from an app.yml file. You can specify its location by setting the CONFIG_FILE environment variable. If not set, it will default to ./configs/app.yml

3- Run the CSV Generation Script: Execute the following command: bash CONFIG_FILE=./app.yml go run ./cmd/gogscsv/main.go

4- The script performs these steps:

• Connects to the Gogs MySQL database using the DSN provided in the configuration.

• Exports users by running:

  SELECT * FROM user WHERE email != '';
  

  and writes the results to a CSV file (path specified in the config).

• Exports organizations by executing a join query and writing the results to a CSV.

• Exports collaborators by running another join query and outputting the CSV.

Summary

1- Restore the Gogs Database Dump:

• Create a dump of your Gogs MySQL database.
• Use init-gogs-db.sh to run a new MySQL container and restore your dump into it.

2- Generate CSV Files:

• Ensure your restored Gogs database is accessible.

• Configure your app.yml file (or set CONFIG_FILE).

• Run the CSV generation Go script:

  CONFIG_FILE=./app.yml go run ./cmd/gogscsv/main.go
  

• This script will produce CSV files for users, organizations, and collaborators, which can then be used with the Gitea API to complete your migration.

Queries for Migration

Users Migration

Export the users with a non-empty email address. Run this query against your Gogs database and export the result to a CSV file:

SELECT * FROM user
WHERE email != '';

Execution:

CONFIG_FILE=./app.yml go run ./cmd/users/main.go

orgs migration

Export results of this query to .csv to migrate the users

SELECT
    u.name,
    ou.org_id,
    uo.name AS owner_name,
    uo.email AS owner_email
FROM org_user ou
INNER JOIN user u ON ou.org_id = u.id
INNER JOIN user uo ON uo.id = ou.is_owner

Execution:

CONFIG_FILE=./app.yml go run ./cmd/org/main.go

repositories migration

You have two options: You can use the API to import them or like I did: manually import using the unadopted button from the UI.

Execution:

CONFIG_FILE=./app.yml go run ./cmd/org/main.go

collaborators migration

Export results of this query to .csv to migrate the repositories

SELECT
    r.name AS repo_name,
    uo.name AS repo_owner,
    u.name AS collaborator_user,
    co.mode,
    CASE
        WHEN co.mode = 1 THEN 'read'
        WHEN co.mode = 2 THEN 'write'
        WHEN co.mode = 3 THEN 'admin'
        WHEN co.mode = 4 THEN 'owner'
        ELSE 'failed'
    END AS permission
FROM collaboration co
INNER JOIN repository r ON r.id = co.repo_id
INNER JOIN user uo ON uo.id = r.owner_id
INNER JOIN user u ON u.id = co.user_id;

Execution:

CONFIG_FILE=./app.yml go run ./cmd/collaborator/main.go