Commit graph

68 commits

Author SHA1 Message Date
Hazelnoot
528c0476f9 add lastActiveDate to updateUserQueue 2025-11-15 14:08:34 -05:00
Hazelnoot
a55649e89a move global services to "global" directory 2025-11-05 19:42:28 -05:00
Hazelnoot
6cceca90f9 use TimeService everywhere in the backend 2025-11-05 19:21:46 -05:00
dakkar
0e985b77d6 fix wss error on shutdown
this is triggered by e2e tests
2025-09-19 13:03:08 +01:00
Hazelnoot
e1639f70df fix null safety in StreamingApiServerService.ts 2025-09-18 20:37:02 -04:00
Hazelnoot
f93693535e disconnect ws error event after promise resolves 2025-09-13 11:41:22 -04:00
Hazelnoot
4f68bbfd52 disconnect startup error handler after establishing connection 2025-09-13 11:41:22 -04:00
Hazelnoot
272f5cc5ab add error handler for connection 2025-09-13 11:41:22 -04:00
Hazelnoot
3770f4a86a disconnect pong() listener when closing connection 2025-09-13 11:41:22 -04:00
Hazelnoot
d18b910791 don't detach error handler until *after* socket close 2025-09-13 11:41:22 -04:00
Hazelnoot
49eb26694a add error handler for ws library 2025-09-13 11:41:22 -04:00
наб
155d2920b6
Dispose of the stream even if it gets closed early 2025-08-11 23:53:30 +02:00
наб
5aceeb31e7
Return request errors for /streaming in-band instead of in HTTP
Closes #591
2025-08-11 23:53:30 +02:00
наб
d954b8d138
StreamingApiServerService: handle websocket errors from client gracefully
$ nc -C tarta.nabijaczleweli.xyz 12122
  GET /streaming?_t=1752534314122 HTTP/1.1
  Host: tarta.nabijaczleweli.xyz:12122
  User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:142.0) Gecko/20100101 Firefox/142.0
  Accept: */*
  Accept-Language: en-GB,en;q=0.5
  Accept-Encoding: gzip, deflate
  Sec-WebSocket-Version: 13
  Sec-WebSocket-Extensions: permessage-deflate
  Sec-WebSocket-Key: AAAAAAAAAAAAAAAAAAAAAA==
  Connection: keep-alive, Upgrade
  Pragma: no-cache
  Cache-Control: no-cache
  Upgrade: websocket
the parameters almost don't matter so long as the server
replies with an upgrade, then press enter once or twice,
at which point the server will have crashed after
  INFO 1  [core nest]             NestFactory: Starting Nest application...
  ERR  *  [core]          Uncaught exception (uncaughtException): RangeError: Invalid WebSocket frame: invalid opcode 13
      at Receiver.getInfo (/srv/Sharkey/node_modules/.pnpm/ws@8.18.1_bufferutil@4.0.9_utf-8-validate@6.0.5/node_modules/ws/lib/receiver.js:311:26)
      at Receiver.startLoop (/srv/Sharkey/node_modules/.pnpm/ws@8.18.1_bufferutil@4.0.9_utf-8-validate@6.0.5/node_modules/ws/lib/receiver.js:155:16)
      at Receiver._write (/srv/Sharkey/node_modules/.pnpm/ws@8.18.1_bufferutil@4.0.9_utf-8-validate@6.0.5/node_modules/ws/lib/receiver.js:94:10)
      at writeOrBuffer (node:internal/streams/writable:572:12)
      at _write (node:internal/streams/writable:501:10)
      at Writable.write (node:internal/streams/writable:510:10)
      at Socket.socketOnData (/srv/Sharkey/node_modules/.pnpm/ws@8.18.1_bufferutil@4.0.9_utf-8-validate@6.0.5/node_modules/ws/lib/websocket.js:1355:35)
      at Socket.emit (node:events:518:28)
      at addChunk (node:internal/streams/readable:561:12)
      at readableAddChunkPushByteMode (node:internal/streams/readable:512:3)
      at Readable.push (node:internal/streams/readable:392:5)
      at TCP.onStreamRead (node:internal/stream_base_commons:189:23) {
    code: 'WS_ERR_INVALID_OPCODE',
    [Symbol(status-code)]: 1002
  }
  INFO *  [core]          The process is going to exit with code 1
  node:events:496
        throw er; // Unhandled 'error' event
        ^

  RangeError: Invalid WebSocket frame: invalid opcode 13
      at Receiver.getInfo (/srv/Sharkey/node_modules/.pnpm/ws@8.18.1_bufferutil@4.0.9_utf-8-validate@6.0.5/node_modules/ws/lib/receiver.js:311:26)
      at Receiver.startLoop (/srv/Sharkey/node_modules/.pnpm/ws@8.18.1_bufferutil@4.0.9_utf-8-validate@6.0.5/node_modules/ws/lib/receiver.js:155:16)
      at Receiver._write (/srv/Sharkey/node_modules/.pnpm/ws@8.18.1_bufferutil@4.0.9_utf-8-validate@6.0.5/node_modules/ws/lib/receiver.js:94:10)
      at writeOrBuffer (node:internal/streams/writable:572:12)
      at _write (node:internal/streams/writable:501:10)
      at Writable.write (node:internal/streams/writable:510:10)
      at Socket.socketOnData (/srv/Sharkey/node_modules/.pnpm/ws@8.18.1_bufferutil@4.0.9_utf-8-validate@6.0.5/node_modules/ws/lib/websocket.js:1355:35)
      at Socket.emit (node:events:518:28)
      at addChunk (node:internal/streams/readable:561:12)
      at readableAddChunkPushByteMode (node:internal/streams/readable:512:3)
      at Readable.push (node:internal/streams/readable:392:5)
      at TCP.onStreamRead (node:internal/stream_base_commons:189:23)
  Emitted 'error' event on WebSocket instance at:
      at Receiver.receiverOnError (/srv/Sharkey/node_modules/.pnpm/ws@8.18.1_bufferutil@4.0.9_utf-8-validate@6.0.5/node_modules/ws/lib/websocket.js:1199:15)
      at Receiver.emit (node:events:518:28)
      at emitErrorNT (node:internal/streams/destroy:170:8)
      at emitErrorCloseNT (node:internal/streams/destroy:129:3)
      at process.processTicksAndRejections (node:internal/process/task_queues:90:21) {
    code: 'WS_ERR_INVALID_OPCODE',
    [Symbol(status-code)]: 1002
  }

  Node.js v22.14.0
   ELIFECYCLE  Command failed with exit code 1.

This works through some reverse proxies (HAProxy, Caddy),
but not through others (Cloudflare, nginx, Apache(?))

Instead, just hang up if the client violates protocol

Fixes https://101010.pl/@nabijaczleweli/114854334401159070
2025-07-21 17:39:41 +02:00
Hazelnoot
be1063238f merge: Enable NestJS shutdown hooks for clean exit (!1145)
View MR for information: https://activitypub.software/TransFem-org/Sharkey/-/merge_requests/1145

Approved-by: dakkar <dakkar@thenautilus.net>
Approved-by: Marie <github@yuugi.dev>
2025-06-29 19:23:55 +00:00
Hazelnoot
d7b94e756d fix websockets not working 2025-06-26 09:32:14 -04:00
Hazelnoot
088fe15be5 improve shutdown logic somewhat 2025-06-25 22:51:32 -04:00
Hazelnoot
168a364162 fix redis listener leak in StreamingApiServerService.ts 2025-06-25 22:51:16 -04:00
Hazelnoot
f67be0a733 fix DI for stream connection 2025-06-23 20:13:24 -04:00
Hazelnoot
6ac37b4d6c lint and type fixes 2025-04-01 20:47:04 -04:00
Hazelnoot
e7632c83dc Merge branch 'develop' into merge/2025-03-24
# Conflicts:
#	packages/backend/src/server/api/StreamingApiServerService.ts
#	packages/backend/src/server/api/stream/Connection.ts
2025-03-31 15:02:52 -04:00
Hazelnoot
47ea8527fd fix wsmessage rate limit definition 2025-03-29 09:44:38 -04:00
Hazelnoot
c41d617e63 limit the number of active connections per client, and limit upgrade requests by user 2025-03-28 11:03:31 -04:00
Hazelnoot
18655386f3 convert streaming rate limit to bucket 2025-03-28 11:03:31 -04:00
Hazelnoot
d8908ef2d8 merge upstream 2025-03-25 16:14:53 -04:00
syuilo
f1f24e39d2
Feat: Chat (#15686)
* wip

* wip

* wip

* wip

* wip

* wip

* Update types.ts

* Create 1742203321812-chat.js

* wip

* wip

* Update room.vue

* Update home.vue

* Update home.vue

* Update ja-JP.yml

* Update index.d.ts

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* Update CHANGELOG.md

* wip

* Update home.vue

* clean up

* Update misskey-js.api.md

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* lint fixes

* lint

* Update UserEntityService.ts

* search

* wip

* 🎨

* wip

* Update home.ownedRooms.vue

* wip

* Update CHANGELOG.md

* Update style.scss

* wip

* improve performance

* improve performance

* Update timeline.test.ts
2025-03-24 21:32:46 +09:00
PrivateGER
c08c8502e3
adjust compression fallback 2025-03-18 14:21:31 +01:00
PrivateGER
aa6bc9c9d0
linter fix 2025-02-26 12:13:13 +01:00
PrivateGER
2b919c4eb0
Optionally enable websocket compression 2025-02-25 22:07:59 +01:00
Hazelnoot
f92fb3bb8c move SkRateLimiterService to correct directory 2025-02-05 11:20:26 -05:00
Hazelnoot
09669d72e7 lookup and cache rate limit factors directly within SkRateLimiterService 2025-02-05 11:20:25 -05:00
Hazelnoot
f6b256620b separate SkRateLimiterService from RateLimiterService and update all usages 2024-12-07 13:13:19 -05:00
dakkar
77144b058c make the cap of activeRateLimitRequests match the rate limit
It's trivial to have more than 128 requests in flight: open a busy
timeline, scroll a bit down, wait for many notes to arrive, scroll to
the top. The frontend will send "subscribe to note" messages for each
new note that it accumulated, all at once. We don't want to shut down
the connection in those common cases!
2024-08-18 17:57:51 +01:00
dakkar
caa0fecdc9 relax websocket rate limits
* the frontend opens 2 websockets at startup (I'm not completely clear
  why), and that `minInterval:1sec` was breaking the second connection
* as the comment says, "catching up" generates many noteSubscribe
  messages
2024-08-18 15:23:45 +01:00
Julia Johannesen
3dd993a76a
Add IP and user ID to connection close message 2024-08-17 14:27:43 -04:00
Julia Johannesen
c5f7dcbb7e
Come up with better limits 2024-08-17 14:17:58 -04:00
Julia Johannesen
9c1c1e9f09
Fix logging stuff 2024-08-17 13:08:46 -04:00
Julia Johannesen
6d3f9503ed
Limit number of rate limit requests 2024-08-16 17:13:20 -04:00
dakkar
4cd44130e0 use the correct remote address
we're doing the same thing that Fastify does in the non-streaming
ServerService
2024-08-16 18:00:50 +01:00
dakkar
311a31da58 rough rate limiting for websockets 2024-08-15 11:35:51 +01:00
tamaina
c1514ce91d (re) update SPDX-FileCopyrightText
Fix  #13290
2024-02-13 15:59:27 +00:00
tamaina
311c2172d7 Revert "update SPDX-FileCopyrightText"
This reverts commit 9b5aeb76d8.
2024-02-13 15:50:11 +00:00
syuilo
9b5aeb76d8 update SPDX-FileCopyrightText 2024-02-12 11:37:45 +09:00
Chocolate Pie
c96bc36fed
Merge pull request from GHSA-7pxq-6xx9-xpgm
* fix: fix improper authorization when accessing with third-party application

* refactor: refactor type definitions

* fix: get rid of unnecessary access limitation

* enhance: サードパーティアプリケーションがWebsocket APIを使えるように

* fix: add missing parentheses

* Revert "fix(backend): add missing kind definition for admin endpoints to improve security"

This reverts commit 5150053275.

* frontend: 翻訳の抜けを訂正, read:adminとwrite:adminはアクセス発行トークンのデフォルトでは非表示にする

* enhance(test): misskey-ghsa-7pxq-6xx9-xpgmに関するテストを追加

* enhance(test): Websocket APIに対するテストも追加

* enhance(refactor): `@/misc/api-permissions.ts`を`misskey-js/permissions`に統合

* fix(frontend): アクセストークン発行UIで全ての権限を有効にした際、管理者用APIへのアクセスも許可してしまう問題を修正

* enhance(backend): Websocketの接続に最低限必要な権限を変更

* fix(backend): `/api/admin/meta`をサードパーティアプリケーションからはアクセスできないように

* fix(backend): エンドポイントにアクセスするために必要な権限を変更

* fix(frontend/locale): Add missing type declaration

* chore: update `misskey-js/src/autogen`

---------

Co-authored-by: tamaina <tamaina@hotmail.co.jp>
2023-12-27 15:08:59 +09:00
おさむのひと
a8ee67cace
Fix: チャンネルのフォロー・アンフォローの反映速度を改善 (#12149)
* チャンネルのフォロー・アンフォローの反映速度を改善

* fix lint

* userFollowingChannelsCacheの場所をCacheServiceからChannelFollowingServiceに移動

---------

Co-authored-by: osamu <46447427+sam-osamu@users.noreply.github.com>
2023-10-27 18:34:02 +09:00
syuilo
6277a5545c
feat: improve tl performance (#11946)
* wip

* wip

* wip

* wip

* wip

* wip

* Update NoteCreateService.ts

* wip

* wip

* wip

* wip

* Update NoteCreateService.ts

* wip

* Update NoteCreateService.ts

* wip

* Update user-notes.ts

* wip

* wip

* wip

* Update NoteCreateService.ts

* wip

* Update timeline.ts

* Update timeline.ts

* Update timeline.ts

* Update timeline.ts

* Update timeline.ts

* wip

* Update timelines.ts

* Update timelines.ts

* Update timelines.ts

* wip

* wip

* wip

* Update timelines.ts

* Update misskey-js.api.md

* Update timelines.ts

* Update timelines.ts

* wip

* wip

* wip

* Update timelines.ts

* wip

* Update timelines.ts

* wip

* test

* Update activitypub.ts

* refactor: UserListJoining -> UserListMembership

* Update NoteCreateService.ts

* wip
2023-10-03 20:26:11 +09:00
syuilo
053da10e94 refactor(backend): update directory structure for models 2023-09-20 11:33:36 +09:00
syuilo
6cf466e5d1
update deps (#11820)
* update deps

* fix

* wip

* wip

* wip

* Update docker-compose.yml.example

* Delete reviewer-lottery.yml

* Update RepositoryModule.ts

* wip

* wip

* clean up

* update deps

* wip

* wip
2023-09-15 14:28:29 +09:00
syuilo
792622aead
refactor: prefix Mi for all entities (#11719)
* wip

* wip

* wip

* wip

* Update RepositoryModule.ts

* wip

* wip

* wip

* Revert "wip"

This reverts commit c1c13b37d2aaf3c65bc148212da302b0eb7868bf.
2023-08-16 17:51:28 +09:00
woxtu
8a6791da3f
refactor(backend): Remove unused injections (#11462)
* Remove unused injections

* Remove unused imports
2023-08-05 10:33:00 +09:00