mistykey/CONTRIBUTING.Sharkey.md

Contribution guide

We're glad you're interested in contributing to Sharkey! In this document you will find the information you need to contribute to the project. Please also read Misskey's contribution guide, CONTRIBUTING.md.

Issues

Before creating an issue, please check the following:

• To avoid duplication, please search for similar issues before creating a new issue.

• Do not use Issues to ask questions or for troubleshooting.

  • Issues should only be used for feature requests, suggestions, and bug tracking.
  • For questions and troubleshooting, join our Discord server or tag the official Sharkey fedi account, @sharkey@sharkey.team.

Warning

Do not close issues that are about to be resolved. It should remain open until a commit that actually resolves it is merged.

Well-known branches

• stable branch is tracking the latest release and used for production purposes.

• develop branch is where we work for the next release.

  • When you create a merge request, target it to this branch.

Creating a Merge Request

Thank you for your merge request! Before creating an MR, please check the following:

• If possible, prefix the title with a keyword that identifies the type of this MR, like fix / refactor / feat / enhance / perf / chore etc

• Make sure that the granularity of this MR is appropriate. Please do not include more than one type of change or interest in a single MR.

• If there is an Issue which will be resolved by this MR, please include a reference to the Issue in the text.

• Check if there are any documents that need to be created or updated due to this change.

• If you have added a feature or fixed a bug, please add a test case if possible.

• Please make sure that tests and Lint are passed in advance. You can run it with pnpm test and pnpm lint. See the testing section for more info.

• If this MR includes new translated strings, or changes to existing translations, make sure you have edited sharkey-locales/en-US.yml and not any of the files under locales/ (the files under locales/ must stay identical to Misskey)

• If this MR includes UI changes, please attach a screenshot in the text.

Release process

(See also the wiki)

Creating a release

Prerequisites

Before creating a release, we must ensure that:

• develop contains exactly the code we want to release.

  • It can be useful to mark MRs we want to release with the for-next-release label.

• We have tested it enough to be confident we can release it to the world.

• The CI pipeline (build, test, lint) passes.

• The backend end-to-end tests (pnpm --filter=backend test:e2e) pass on your machine.

• package.json and packages/misskey-js/package.json on develop have a "version": "2027.12.0-dev" or something similarly non-prod-looking.

• The changelogs contain all the changes we want to announce.

To release

To create a release, we must:

1. Create a new Branch based on develop to change the version to a prod-looking one (e.g. 2027.12.1).

   • Try to avoid using the same version as Misskey, both to reduce confusion, and because (unlike branches) tags are not scoped by remote and will confuse multi-remote clones.

2. Create an MR to merge the new branch into stable.

3. Once that MR is merged, go to https://activitypub.software/TransFem-org/Sharkey/-/releases and create a new release.

• For the tag, use the same version you just set on stable (e.g. 2027.12.1).

• Make sure the tag will be created on stable.

  • For the release name, again use the version (e.g. 2027.12.1).

  • For the release notes, copy the changelogs.

4. Wait for all the pipelines to complete.

   • In the container registry you should get (of course with the right version):

     • latest
       • 2027.12.1-amd64
       • 2027.12.1-arm64

5. Announce the release on the official account!

Post release

After creating a release, we must:

• Branch off develop, merge stable into that new branch, change the version to the next number (e.g. 2028.1.0-dev), create a MR for this branch, and get it merged.

Hotfixes / security releases

Sometimes we need to release changes to the latest stable release, without whatever has been merged into develop. For example, a security fix.

In these cases:

1. Create a branch off stable - let's call it hotfix/2027.12.2, for example - and change the version number on this branch.

2. Create branches off stable, one per fix (like normal feature / bugfix branches, but starting from the released code), and send MRs targeting hotfix/2027.12.2.

3. Once all the fixes have been merged into hotfix/2027.12.2, create a MR targeting stable.

4. Now carry on through the normal release process (from step 3 of the "To release" section).

Icon Font (Shark Font)

Sharkey has its own Icon Font called Shark Font which can be found at https://activitypub.software/TransFem-org/shark-font

Build instructions can all be found over there in the README.

If you have an icon suggestion or want to add an Icon please open an issue/merge request over at that repo.

When updating the font, make sure to copy all generated files from the dest folder into packages/backend/assets/fonts/sharkey-icons.

For the CSS, copy the file content and replace the old content in style.css and for the WOFF, TTF and SVG simply replace them.

Development

Accessing source code

In order to submit code changes, you will need to create a fork of the main repository. This can be done via the GitLab UI, by pressing the "Fork" button while signed into an activitypub.software GitLab account.

Once you have created a fork, you should clone it locally and update submodules using Git. For example, to clone using SSH, use the following commands, replacing "<YOUR_USERNAME>" with your GitLab username:

git clone git@activitypub.software:<YOUR_USERNAME>/Sharkey.git
git submodule update --init

Environment setup

Before developing, you should set up a testing environment. You can do this using Docker via the Docker Compose plugin. You will also need to have pnpm installed.

(You may wish to perform this setup using system-wide software installed separately, e.g. via a package manager, or using Devcontainer. Both are possible, but they will require manual setup that will not be covered in this document.)

First, you will need to copy .config/docker_example.env to .config/docker.env. This file will contain configurations for the PostgreSQL database, such as username and password. You may set these as you wish. You will also need to copy .config/example.yml to .config/default.yml. This file will contain configurations for Sharkey. Ensure that the username and password in the db: section match the ones set in docker.env.

Now, use the following command to start a local database container:

docker compose -f compose.local-db.yml up -d

This will run a local PostgreSQL database server in the background. (To stop the database, run docker compose -f compose.local-db.yml down.)

Once the database is active, run the following commands:

pnpm build
pnpm migrate

This will build Sharkey and perform database migrations. After finishing the migration, the database will be ready for use.

Start developing

After making code changes, you can run Sharkey using the following command:

pnpm dev

• Checks server-side source files and automatically builds them if they are modified. Automatically starts the server process(es).

• Vite HMR (just the vite command) is available. The behavior may be different from production.

• Service Worker is watched by esbuild.

• Sharkey is served on the port configured with port in .config/default.yml. If you have not changed it from the default, you can access it at http://localhost:3000.

Testing

(See also Misskey's docs about testing).

To run many of the tests, you need a dedicated database. To set this up:

1. Start PostgreSQL and Redis

   • This is covered in the environment setup section.

2. Create the test configuration file, by copying .config/test-example.yml to .config/test.yml.

   cp .config/test-example.yml .config/test.yml
   

3. Start the database container:

   docker compose -f packages/backend/test/compose.yml up -d
   

Now you can run pnpm test and pnpm --filter=backend test:e2e to run the tests.

To stop the database container, run docker compose -f packages/backend/test/compose.yml up -d down

Environment Variables

• MISSKEY_CONFIG_DIR changes the directory where config files are searched. Defaults to .config/ at the top of the repository.
• MISSKEY_CONFIG_YML changes the configuration file name. Defaults to default.yml (e.g. you can set MISSKEY_CONFIG_YML=2nd.yml to load .config/2nd.yml).
• MISSKEY_WEBFINGER_USE_HTTP if set to true, WebFinger requests will be http instead of https, useful for testing federation between servers in localhost.

Warning

Never use MISSKEY_WEBFINGER_USE_HTTP for a real instance in production, as it will expose user information to the network.

Continuous integration

Sharkey uses GitLab CI for executing automated tests.

Configuration files are located in .gitlab-ci.yml.

Merging from Misskey into Sharkey

Make sure you have both remotes in the same clone. To do this, run the following command to add Misskey's remote to your local clone:

git remote add misskey https://github.com/misskey-dev/misskey.git

Then, to perform the merge:

git remote update   # Update branches to track the new remote.
git checkout develop   # Checkout Sharkey's develop branch.
git checkout -m merge/$(date +%Y-%m-%d)   # Create/switch to a merge branch for the current date.
git merge --no-ff misskey/develop   # Merge from Misskey's develop branch, forcing a merge commit.

Fix conflicts and commit! Conflicts in pnpm-lock.yaml can usually be fixed by running pnpm install - it detects conflict markers and seems to do a decent job.

After that commit, do all the extra work, on the same branch:

• Copy all changes (commit after each step):

  • in packages/backend/src/core/activitypub/models/ApNoteService.ts, from createNote to updateNote
  • from packages/backend/src/core/NoteCreateService.ts to packages/backend/src/core/NoteEditService.ts
  • from packages/backend/src/server/api/endpoints/notes/create.ts to packages/backend/src/server/api/endpoints/notes/edit.ts
  • from MK note components to SK note components (if sensible)
    • from packages/frontend/src/components/MkNote.vue to packages/frontend/src/components/SkNote.vue
    • from packages/frontend/src/components/MkNoteDetailed.vue to packages/frontend/src/components/SkNoteDetailed.vue
    • from packages/frontend/src/components/MkNoteHeader.vue to packages/frontend/src/components/SkNoteHeader.vue
    • from packages/frontend/src/components/MkNoteSimple.vue to packages/frontend/src/components/SkNoteSimple.vue
    • from packages/frontend/src/components/MkNoteSub.vue to packages/frontend/src/components/SkNoteSub.vue
  • from MK note components to Dynamic note components (if the public signature changed)
    • from packages/frontend/src/components/MkNote.vue to packages/frontend/src/components/DynamicNote.vue
    • from packages/frontend/src/components/MkNoteDetailed.vue to packages/frontend/src/components/DynamicNoteDetailed.vue
    • from packages/frontend/src/components/MkNoteSimple.vue to packages/frontend/src/components/DynamicNoteSimple.vue
  • from the global timeline to the bubble timeline
    • packages/backend/src/server/api/stream/channels/global-timeline.ts
    • packages/backend/src/server/api/stream/channels/bubble-timeline.ts
    • packages/frontend/src/timelines.ts
    • packages/frontend/src/components/MkTimeline.vue
    • packages/frontend/src/pages/timeline.vue
    • packages/frontend/src/ui/deck/tl-column.vue
    • packages/frontend/src/widgets/WidgetTimeline.vue
  • from packages/backend/src/queue/processors/InboxProcessorService.ts to packages/backend/src/core/UpdateInstanceQueue.ts, where updateInstanceQueue is impacted
  • from .config/example.yml to .config/ci.yml and chart/files/default.yml
  • in packages/backend/src/core/MfmService.ts, from toHtml to toMastoApiHtml
  • from verifyLink in packages/backend/src/core/activitypub/models/ApPersonService.ts to verifyFieldLinks in packages/backend/src/misc/verify-field-link.ts (if sensible)

• Check for changes that may require additional work:

  • If there have been any changes to the federated user data (the renderPerson function in packages/backend/src/core/activitypub/ApRendererService.ts), make sure that the set of fields in userNeedsPublishing and profileNeedsPublishing in packages/backend/src/server/api/endpoints/i/update.ts are still correct.
  • Check for any new instances of any memory cache class. (MemoryKVCache, MemorySingleCache, RedisKVCache, RedisSingleCache, and QuantumKVCache are the current ones.) These can usually be kept as-is, but all instances must be managed by CacheManagementService. The conversion is easy:
    1. Make sure that CacheManagementService is available. In most cases, it can be injected through DI. (it's in the GlobalModule which should be available everywhere.)
    2. Find where the cache is constructed. If it's a field initializer, then move it to the constructor (splitting declaration and initialization.)
    3. Replace the new Whatever() statement with a call to cacheManagementService.createWhatever(). Arguments can be kept as-is, but remove any references to Redis, InternalEventService, or TimeService. (these are provided by CacheManagementService directly.)
    4. Remove any calls to dispose() the cache. Disposal is managed by CacheManagementService, so attempting to call any dispose or onApplicationShutdown method will produce a type error.
  • Check for any new calls to native time functions:
    • Date.now() - replace with this.timeService.now. Inject TimeService via DI if it's not already available.
    • new Date() - if there's a value passed in, then leave it. But the no-args constructor should be replaced with this.timeService.date. Inject TimeService via DI if it's not already available.
    • setTimeout - migrate to this.timeService.startTimer or this.timeService.startPromiseTimer. The parameters should be the same, but the return type is different. You may need to replace some NodeJS.Timeout types with this.timerHandle. Inject TimeService via DI if it's not already available.
    • setInterval - migrate to this.timeService.startTimer. Migration is mostly the same as setTimeout, but with one major difference: You must add { repeated: true } as the final option parameter. If this is omitted, the code will compile but the interval will only fire once!
  • Check for any new Chart subclasses, and make sure to inject TimeService and implement getCurrentDate.
  • Check for any new Channel subclasses and add all missing DI parameters.

• Check the changes against our develop branch (git diff develop) and against Misskey's develop branch (git diff misskey/develop).

• Re-generate misskey-js (pnpm build-misskey-js-with-types) and commit.

• Re-generate locales (pnpm run build-assets) and commit.

• Build the frontend with this command:

  rm -rf built/
  NODE_ENV=development pnpm --filter=frontend --filter=frontend-embed --filter=frontend-shared build
  

• Make sure there aren't any new ti-* classes (Tabler Icons), and replace them with appropriate ph-* ones (Phosphor Icons) in vite.replaceicons.ts.

  • This command should show you want to change: grep -ohrP '(?<=["'\''](ti )?)(ti-(?!fw)[\w\-]+)' --exclude \*.map -- built/ | sort -u.

  • NOTE: ti-fw is a special class that's defined by Misskey, leave it alone.

  • After every change, re-build the frontend and check again, until there are no more ti-* classes in the built files.

  • Commit!

• Double-check the new migration, that they won't conflict with our db changes: git diff develop -- packages/backend/migration/

• Run pnpm clean; pnpm build.

• Run tests pnpm test; pnpm --filter backend test:e2e (requires a test database, see above) and fix them all (the e2e tests randomly fail with weird errors like relation "users" does not exist, run them again if that happens).

• Run lint pnpm --filter=backend --filter=frontend-shared lint + pnpm --filter=frontend --filter=frontend-embed eslint and fix all the problems.

Then push and open a Merge Request.

Memory Caches

Sharkey offers multiple memory cache implementations, each meant for a different use case. The following table compares the available options:

Cache	Type	Consistency	Persistence	Data Source	Cardinality	Eviction	Description
MemoryKVCache	Key-Value	None	None	Caller	Single	Lifetime	Implements a basic in-memory Key-Value store. The implementation is entirely synchronous, except for user-provided data sources.
MemorySingleCache	Single	None	None	Caller	Single	Lifetime	Implements a basic in-memory Single Value store. The implementation is entirely synchronous, except for user-provided data sources.
RedisKVCache	Key-Value	Eventual	Redis	Callback	Single	Lifetime	Extends MemoryKVCache with Redis-backed persistence and a pre-defined callback data source. This provides eventual consistency guarantees based on the memory cache lifetime.
RedisSingleCache	Single	Eventual	Redis	Callback	Single	Lifetime	Extends MemorySingleCache with Redis-backed persistence and a pre-defined callback data source. This provides eventual consistency guarantees based on the memory cache lifetime.
QuantumKVCache	Key-Value	Immediate	None	Callback	Multiple	Lifetime	Combines MemoryKVCache with a pre-defined callback data source and immediate consistency via Redis sync events. The implementation offers multi-item batch overloads for efficient bulk operations. This is the recommended cache implementation for most use cases.

Key-Value caches store multiple entries per cache, while Single caches store a single value that can be accessed directly. Consistency refers to the consistency of cached data between different processes in the instance cluster: "None" means no consistency guarantees, "Eventual" caches will gradually become consistent after some unknown time, and "Immediate" consistency ensures accurate data ASAP after the update. Caches with persistence can retain their data after a reboot through an external service such as Redis. If a data source is supported, then this allows the cache to directly load missing data in response to a fetch. "Caller" data sources are passed into the fetch method(s) directly, while "Callback" sources are passed in as a function when the cache is first initialized. The cardinality of a cache refers to the number of items that can be updated in a single operation, and eviction, finally, is the method that the cache uses to evict stale data.

Selecting a cache implementation

For most cache uses, QuantumKVCache should be considered first. It offers strong consistency guarantees, multiple cardinality, and a cleaner API surface than the older caches.

An alternate cache implementation should be considered if any of the following apply:

• The data is particularly slow to calculate or difficult to access. In these cases, either RedisKVCache or RedisSingleCache should be considered.

• If stale data is acceptable, then consider MemoryKVCache or MemorySingleCache. These synchronous implementations have much less overhead than the other options.

• There is only one data item, or all data items must be fetched together. Using MemorySingleCache or RedisSingleCache could provide a cleaner implementation without resorting to hacks like a fixed key.

• It's necessary to use null as a data value. QuantumKVCache does not allow null values, and thus another option should be chosen.