merge: Avoid more N+1 queries in NoteEntityService and UserEntityService (!1099)

View MR for information: https://activitypub.software/TransFem-org/Sharkey/-/merge_requests/1099

Approved-by: dakkar <dakkar@thenautilus.net>
Approved-by: Marie <github@yuugi.dev>
This commit is contained in:
Hazelnoot 2025-06-12 15:42:39 +00:00
commit 55551a5a8a
67 changed files with 2684 additions and 683 deletions

View file

@ -622,6 +622,35 @@ marginはそのコンポーネントを使う側が設定する
### indexというファイル名を使うな ### indexというファイル名を使うな
ESMではディレクトリインポートは廃止されているのと、ディレクトリインポートせずともファイル名が index だと何故か一部のライブラリ?でディレクトリインポートだと見做されてエラーになる ESMではディレクトリインポートは廃止されているのと、ディレクトリインポートせずともファイル名が index だと何故か一部のライブラリ?でディレクトリインポートだと見做されてエラーになる
### Memory Caches
Sharkey offers multiple memory cache implementations, each meant for a different use case.
The following table compares the available options:
| Cache | Type | Consistency | Persistence | Data Source | Cardinality | Eviction | Description |
|---------------------|-----------|-------------|-------------|-------------|-------------|----------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `MemoryKVCache` | Key-Value | None | None | Caller | Single | Lifetime | Implements a basic in-memory Key-Value store. The implementation is entirely synchronous, except for user-provided data sources. |
| `MemorySingleCache` | Single | None | None | Caller | Single | Lifetime | Implements a basic in-memory Single Value store. The implementation is entirely synchronous, except for user-provided data sources. |
| `RedisKVCache` | Key-Value | Eventual | Redis | Callback | Single | Lifetime | Extends `MemoryKVCache` with Redis-backed persistence and a pre-defined callback data source. This provides eventual consistency guarantees based on the memory cache lifetime. |
| `RedisSingleCache` | Single | Eventual | Redis | Callback | Single | Lifetime | Extends `MemorySingleCache` with Redis-backed persistence and a pre-defined callback data source. This provides eventual consistency guarantees based on the memory cache lifetime. |
| `QuantumKVCache` | Key-Value | Immediate | None | Callback | Multiple | Lifetime | Combines `MemoryKVCache` with a pre-defined callback data source and immediate consistency via Redis sync events. The implementation offers multi-item batch overloads for efficient bulk operations. **This is the recommended cache implementation for most use cases.** |
Key-Value caches store multiple entries per cache, while Single caches store a single value that can be accessed directly.
Consistency refers to the consistency of cached data between different processes in the instance cluster: "None" means no consistency guarantees, "Eventual" caches will gradually become consistent after some unknown time, and "Immediate" consistency ensures accurate data ASAP after the update.
Caches with persistence can retain their data after a reboot through an external service such as Redis.
If a data source is supported, then this allows the cache to directly load missing data in response to a fetch.
"Caller" data sources are passed into the fetch method(s) directly, while "Callback" sources are passed in as a function when the cache is first initialized.
The cardinality of a cache refers to the number of items that can be updated in a single operation, and eviction, finally, is the method that the cache uses to evict stale data.
#### Selecting a cache implementation
For most cache uses, `QuantumKVCache` should be considered first.
It offers strong consistency guarantees, multiple cardinality, and a cleaner API surface than the older caches.
An alternate cache implementation should be considered if any of the following apply:
* The data is particularly slow to calculate or difficult to access. In these cases, either `RedisKVCache` or `RedisSingleCache` should be considered.
* If stale data is acceptable, then consider `MemoryKVCache` or `MemorySingleCache`. These synchronous implementations have much less overhead than the other options.
* There is only one data item, or all data items must be fetched together. Using `MemorySingleCache` or `RedisSingleCache` could provide a cleaner implementation without resorting to hacks like a fixed key.
## CSS Recipe ## CSS Recipe
### Lighten CSS vars ### Lighten CSS vars

View file

@ -26,6 +26,7 @@ import PerUserFollowingChart from '@/core/chart/charts/per-user-following.js';
import { SystemAccountService } from '@/core/SystemAccountService.js'; import { SystemAccountService } from '@/core/SystemAccountService.js';
import { RoleService } from '@/core/RoleService.js'; import { RoleService } from '@/core/RoleService.js';
import { AntennaService } from '@/core/AntennaService.js'; import { AntennaService } from '@/core/AntennaService.js';
import { CacheService } from '@/core/CacheService.js';
@Injectable() @Injectable()
export class AccountMoveService { export class AccountMoveService {
@ -68,6 +69,7 @@ export class AccountMoveService {
private systemAccountService: SystemAccountService, private systemAccountService: SystemAccountService,
private roleService: RoleService, private roleService: RoleService,
private antennaService: AntennaService, private antennaService: AntennaService,
private readonly cacheService: CacheService,
) { ) {
} }
@ -107,12 +109,10 @@ export class AccountMoveService {
this.globalEventService.publishMainStream(src.id, 'meUpdated', iObj); this.globalEventService.publishMainStream(src.id, 'meUpdated', iObj);
// Unfollow after 24 hours // Unfollow after 24 hours
const followings = await this.followingsRepository.findBy({ const followings = await this.cacheService.userFollowingsCache.fetch(src.id);
followerId: src.id, this.queueService.createDelayedUnfollowJob(Array.from(followings.keys()).map(followeeId => ({
});
this.queueService.createDelayedUnfollowJob(followings.map(following => ({
from: { id: src.id }, from: { id: src.id },
to: { id: following.followeeId }, to: { id: followeeId },
})), process.env.NODE_ENV === 'test' ? 10000 : 1000 * 60 * 60 * 24); })), process.env.NODE_ENV === 'test' ? 10000 : 1000 * 60 * 60 * 24);
await this.postMoveProcess(src, dst); await this.postMoveProcess(src, dst);
@ -138,11 +138,9 @@ export class AccountMoveService {
// follow the new account // follow the new account
const proxy = await this.systemAccountService.fetch('proxy'); const proxy = await this.systemAccountService.fetch('proxy');
const followings = await this.followingsRepository.findBy({ const followings = await this.cacheService.userFollowersCache.fetch(src.id)
followeeId: src.id, .then(fs => Array.from(fs.values())
followerHost: IsNull(), // follower is local .filter(f => f.followerHost == null && f.followerId !== proxy.id));
followerId: Not(proxy.id),
});
const followJobs = followings.map(following => ({ const followJobs = followings.map(following => ({
from: { id: following.followerId }, from: { id: following.followerId },
to: { id: dst.id }, to: { id: dst.id },
@ -318,9 +316,9 @@ export class AccountMoveService {
await this.usersRepository.decrement({ id: In(localFollowerIds) }, 'followingCount', 1); await this.usersRepository.decrement({ id: In(localFollowerIds) }, 'followingCount', 1);
// Decrease follower counts of local followees by 1. // Decrease follower counts of local followees by 1.
const oldFollowings = await this.followingsRepository.findBy({ followerId: oldAccount.id }); const oldFollowings = await this.cacheService.userFollowingsCache.fetch(oldAccount.id);
if (oldFollowings.length > 0) { if (oldFollowings.size > 0) {
await this.usersRepository.decrement({ id: In(oldFollowings.map(following => following.followeeId)) }, 'followersCount', 1); await this.usersRepository.decrement({ id: In(Array.from(oldFollowings.keys())) }, 'followersCount', 1);
} }
// Update instance stats by decreasing remote followers count by the number of local followers who were following the old account. // Update instance stats by decreasing remote followers count by the number of local followers who were following the old account.

View file

@ -130,7 +130,8 @@ export class AntennaService implements OnApplicationShutdown {
} }
if (note.visibility === 'followers') { if (note.visibility === 'followers') {
const isFollowing = Object.hasOwn(await this.cacheService.userFollowingsCache.fetch(antenna.userId), note.userId); const followings = await this.cacheService.userFollowingsCache.fetch(antenna.userId);
const isFollowing = followings.has(note.userId);
if (!isFollowing && antenna.userId !== note.userId) return false; if (!isFollowing && antenna.userId !== note.userId) return false;
} }

View file

@ -5,14 +5,16 @@
import { Inject, Injectable } from '@nestjs/common'; import { Inject, Injectable } from '@nestjs/common';
import * as Redis from 'ioredis'; import * as Redis from 'ioredis';
import { IsNull } from 'typeorm'; import { In, IsNull } from 'typeorm';
import type { BlockingsRepository, FollowingsRepository, MutingsRepository, RenoteMutingsRepository, MiUserProfile, UserProfilesRepository, UsersRepository, MiFollowing, MiNote } from '@/models/_.js'; import type { BlockingsRepository, FollowingsRepository, MutingsRepository, RenoteMutingsRepository, MiUserProfile, UserProfilesRepository, UsersRepository, MiNote, MiFollowing } from '@/models/_.js';
import { MemoryKVCache, RedisKVCache } from '@/misc/cache.js'; import { MemoryKVCache, RedisKVCache } from '@/misc/cache.js';
import { QuantumKVCache } from '@/misc/QuantumKVCache.js';
import type { MiLocalUser, MiUser } from '@/models/User.js'; import type { MiLocalUser, MiUser } from '@/models/User.js';
import { DI } from '@/di-symbols.js'; import { DI } from '@/di-symbols.js';
import { UserEntityService } from '@/core/entities/UserEntityService.js'; import { UserEntityService } from '@/core/entities/UserEntityService.js';
import { bindThis } from '@/decorators.js'; import { bindThis } from '@/decorators.js';
import type { GlobalEvents } from '@/core/GlobalEventService.js'; import type { InternalEventTypes } from '@/core/GlobalEventService.js';
import { InternalEventService } from '@/core/InternalEventService.js';
import type { OnApplicationShutdown } from '@nestjs/common'; import type { OnApplicationShutdown } from '@nestjs/common';
export interface FollowStats { export interface FollowStats {
@ -27,7 +29,7 @@ export interface CachedTranslation {
text: string | undefined; text: string | undefined;
} }
interface CachedTranslationEntity { export interface CachedTranslationEntity {
l?: string; l?: string;
t?: string; t?: string;
u?: number; u?: number;
@ -39,14 +41,16 @@ export class CacheService implements OnApplicationShutdown {
public localUserByNativeTokenCache: MemoryKVCache<MiLocalUser | null>; public localUserByNativeTokenCache: MemoryKVCache<MiLocalUser | null>;
public localUserByIdCache: MemoryKVCache<MiLocalUser>; public localUserByIdCache: MemoryKVCache<MiLocalUser>;
public uriPersonCache: MemoryKVCache<MiUser | null>; public uriPersonCache: MemoryKVCache<MiUser | null>;
public userProfileCache: RedisKVCache<MiUserProfile>; public userProfileCache: QuantumKVCache<MiUserProfile>;
public userMutingsCache: RedisKVCache<Set<string>>; public userMutingsCache: QuantumKVCache<Set<string>>;
public userBlockingCache: RedisKVCache<Set<string>>; public userBlockingCache: QuantumKVCache<Set<string>>;
public userBlockedCache: RedisKVCache<Set<string>>; // NOTE: 「被」Blockキャッシュ public userBlockedCache: QuantumKVCache<Set<string>>; // NOTE: 「被」Blockキャッシュ
public renoteMutingsCache: RedisKVCache<Set<string>>; public renoteMutingsCache: QuantumKVCache<Set<string>>;
public userFollowingsCache: RedisKVCache<Record<string, Pick<MiFollowing, 'withReplies'> | undefined>>; public userFollowingsCache: QuantumKVCache<Map<string, Omit<MiFollowing, 'isFollowerHibernated'>>>;
private readonly userFollowStatsCache = new MemoryKVCache<FollowStats>(1000 * 60 * 10); // 10 minutes public userFollowersCache: QuantumKVCache<Map<string, Omit<MiFollowing, 'isFollowerHibernated'>>>;
private readonly translationsCache: RedisKVCache<CachedTranslationEntity>; public hibernatedUserCache: QuantumKVCache<boolean>;
protected userFollowStatsCache = new MemoryKVCache<FollowStats>(1000 * 60 * 10); // 10 minutes
protected translationsCache: RedisKVCache<CachedTranslationEntity>;
constructor( constructor(
@Inject(DI.redis) @Inject(DI.redis)
@ -74,6 +78,7 @@ export class CacheService implements OnApplicationShutdown {
private followingsRepository: FollowingsRepository, private followingsRepository: FollowingsRepository,
private userEntityService: UserEntityService, private userEntityService: UserEntityService,
private readonly internalEventService: InternalEventService,
) { ) {
//this.onMessage = this.onMessage.bind(this); //this.onMessage = this.onMessage.bind(this);
@ -82,58 +87,148 @@ export class CacheService implements OnApplicationShutdown {
this.localUserByIdCache = new MemoryKVCache<MiLocalUser>(1000 * 60 * 5); // 5m this.localUserByIdCache = new MemoryKVCache<MiLocalUser>(1000 * 60 * 5); // 5m
this.uriPersonCache = new MemoryKVCache<MiUser | null>(1000 * 60 * 5); // 5m this.uriPersonCache = new MemoryKVCache<MiUser | null>(1000 * 60 * 5); // 5m
this.userProfileCache = new RedisKVCache<MiUserProfile>(this.redisClient, 'userProfile', { this.userProfileCache = new QuantumKVCache(this.internalEventService, 'userProfile', {
lifetime: 1000 * 60 * 30, // 30m lifetime: 1000 * 60 * 30, // 30m
memoryCacheLifetime: 1000 * 60, // 1m
fetcher: (key) => this.userProfilesRepository.findOneByOrFail({ userId: key }), fetcher: (key) => this.userProfilesRepository.findOneByOrFail({ userId: key }),
toRedisConverter: (value) => JSON.stringify(value), bulkFetcher: userIds => this.userProfilesRepository.findBy({ userId: In(userIds) }).then(ps => ps.map(p => [p.userId, p])),
fromRedisConverter: (value) => JSON.parse(value), // TODO: date型の考慮
}); });
this.userMutingsCache = new RedisKVCache<Set<string>>(this.redisClient, 'userMutings', { this.userMutingsCache = new QuantumKVCache<Set<string>>(this.internalEventService, 'userMutings', {
lifetime: 1000 * 60 * 30, // 30m lifetime: 1000 * 60 * 30, // 30m
memoryCacheLifetime: 1000 * 60, // 1m
fetcher: (key) => this.mutingsRepository.find({ where: { muterId: key }, select: ['muteeId'] }).then(xs => new Set(xs.map(x => x.muteeId))), fetcher: (key) => this.mutingsRepository.find({ where: { muterId: key }, select: ['muteeId'] }).then(xs => new Set(xs.map(x => x.muteeId))),
toRedisConverter: (value) => JSON.stringify(Array.from(value)), bulkFetcher: muterIds => this.mutingsRepository
fromRedisConverter: (value) => new Set(JSON.parse(value)), .createQueryBuilder('muting')
.select('"muting"."muterId"', 'muterId')
.addSelect('array_agg("muting"."muteeId")', 'muteeIds')
.where({ muterId: In(muterIds) })
.groupBy('muting.muterId')
.getRawMany<{ muterId: string, muteeIds: string[] }>()
.then(ms => ms.map(m => [m.muterId, new Set(m.muteeIds)])),
}); });
this.userBlockingCache = new RedisKVCache<Set<string>>(this.redisClient, 'userBlocking', { this.userBlockingCache = new QuantumKVCache<Set<string>>(this.internalEventService, 'userBlocking', {
lifetime: 1000 * 60 * 30, // 30m lifetime: 1000 * 60 * 30, // 30m
memoryCacheLifetime: 1000 * 60, // 1m
fetcher: (key) => this.blockingsRepository.find({ where: { blockerId: key }, select: ['blockeeId'] }).then(xs => new Set(xs.map(x => x.blockeeId))), fetcher: (key) => this.blockingsRepository.find({ where: { blockerId: key }, select: ['blockeeId'] }).then(xs => new Set(xs.map(x => x.blockeeId))),
toRedisConverter: (value) => JSON.stringify(Array.from(value)), bulkFetcher: blockerIds => this.blockingsRepository
fromRedisConverter: (value) => new Set(JSON.parse(value)), .createQueryBuilder('blocking')
.select('"blocking"."blockerId"', 'blockerId')
.addSelect('array_agg("blocking"."blockeeId")', 'blockeeIds')
.where({ blockerId: In(blockerIds) })
.groupBy('blocking.blockerId')
.getRawMany<{ blockerId: string, blockeeIds: string[] }>()
.then(ms => ms.map(m => [m.blockerId, new Set(m.blockeeIds)])),
}); });
this.userBlockedCache = new RedisKVCache<Set<string>>(this.redisClient, 'userBlocked', { this.userBlockedCache = new QuantumKVCache<Set<string>>(this.internalEventService, 'userBlocked', {
lifetime: 1000 * 60 * 30, // 30m lifetime: 1000 * 60 * 30, // 30m
memoryCacheLifetime: 1000 * 60, // 1m
fetcher: (key) => this.blockingsRepository.find({ where: { blockeeId: key }, select: ['blockerId'] }).then(xs => new Set(xs.map(x => x.blockerId))), fetcher: (key) => this.blockingsRepository.find({ where: { blockeeId: key }, select: ['blockerId'] }).then(xs => new Set(xs.map(x => x.blockerId))),
toRedisConverter: (value) => JSON.stringify(Array.from(value)), bulkFetcher: blockeeIds => this.blockingsRepository
fromRedisConverter: (value) => new Set(JSON.parse(value)), .createQueryBuilder('blocking')
.select('"blocking"."blockeeId"', 'blockeeId')
.addSelect('array_agg("blocking"."blockeeId")', 'blockeeIds')
.where({ blockeeId: In(blockeeIds) })
.groupBy('blocking.blockeeId')
.getRawMany<{ blockeeId: string, blockerIds: string[] }>()
.then(ms => ms.map(m => [m.blockeeId, new Set(m.blockerIds)])),
}); });
this.renoteMutingsCache = new RedisKVCache<Set<string>>(this.redisClient, 'renoteMutings', { this.renoteMutingsCache = new QuantumKVCache<Set<string>>(this.internalEventService, 'renoteMutings', {
lifetime: 1000 * 60 * 30, // 30m lifetime: 1000 * 60 * 30, // 30m
memoryCacheLifetime: 1000 * 60, // 1m
fetcher: (key) => this.renoteMutingsRepository.find({ where: { muterId: key }, select: ['muteeId'] }).then(xs => new Set(xs.map(x => x.muteeId))), fetcher: (key) => this.renoteMutingsRepository.find({ where: { muterId: key }, select: ['muteeId'] }).then(xs => new Set(xs.map(x => x.muteeId))),
toRedisConverter: (value) => JSON.stringify(Array.from(value)), bulkFetcher: muterIds => this.renoteMutingsRepository
fromRedisConverter: (value) => new Set(JSON.parse(value)), .createQueryBuilder('muting')
.select('"muting"."muterId"', 'muterId')
.addSelect('array_agg("muting"."muteeId")', 'muteeIds')
.where({ muterId: In(muterIds) })
.groupBy('muting.muterId')
.getRawMany<{ muterId: string, muteeIds: string[] }>()
.then(ms => ms.map(m => [m.muterId, new Set(m.muteeIds)])),
}); });
this.userFollowingsCache = new RedisKVCache<Record<string, Pick<MiFollowing, 'withReplies'> | undefined>>(this.redisClient, 'userFollowings', { this.userFollowingsCache = new QuantumKVCache<Map<string, Omit<MiFollowing, 'isFollowerHibernated'>>>(this.internalEventService, 'userFollowings', {
lifetime: 1000 * 60 * 30, // 30m lifetime: 1000 * 60 * 30, // 30m
memoryCacheLifetime: 1000 * 60, // 1m fetcher: (key) => this.followingsRepository.findBy({ followerId: key }).then(xs => new Map(xs.map(f => [f.followeeId, f]))),
fetcher: (key) => this.followingsRepository.find({ where: { followerId: key }, select: ['followeeId', 'withReplies'] }).then(xs => { bulkFetcher: followerIds => this.followingsRepository
const obj: Record<string, Pick<MiFollowing, 'withReplies'> | undefined> = {}; .findBy({ followerId: In(followerIds) })
for (const x of xs) { .then(fs => fs
obj[x.followeeId] = { withReplies: x.withReplies }; .reduce((groups, f) => {
let group = groups.get(f.followerId);
if (!group) {
group = new Map();
groups.set(f.followerId, group);
} }
return obj; group.set(f.followeeId, f);
}), return groups;
toRedisConverter: (value) => JSON.stringify(value), }, new Map<string, Map<string, Omit<MiFollowing, 'isFollowerHibernated'>>>)),
fromRedisConverter: (value) => JSON.parse(value), });
this.userFollowersCache = new QuantumKVCache<Map<string, Omit<MiFollowing, 'isFollowerHibernated'>>>(this.internalEventService, 'userFollowers', {
lifetime: 1000 * 60 * 30, // 30m
fetcher: followeeId => this.followingsRepository.findBy({ followeeId: followeeId }).then(xs => new Map(xs.map(x => [x.followerId, x]))),
bulkFetcher: followeeIds => this.followingsRepository
.findBy({ followeeId: In(followeeIds) })
.then(fs => fs
.reduce((groups, f) => {
let group = groups.get(f.followeeId);
if (!group) {
group = new Map();
groups.set(f.followeeId, group);
}
group.set(f.followerId, f);
return groups;
}, new Map<string, Map<string, Omit<MiFollowing, 'isFollowerHibernated'>>>)),
});
this.hibernatedUserCache = new QuantumKVCache<boolean>(this.internalEventService, 'hibernatedUsers', {
lifetime: 1000 * 60 * 30, // 30m
fetcher: async userId => {
const { isHibernated } = await this.usersRepository.findOneOrFail({
where: { id: userId },
select: { isHibernated: true },
});
return isHibernated;
},
bulkFetcher: async userIds => {
const results = await this.usersRepository.find({
where: { id: In(userIds) },
select: { id: true, isHibernated: true },
});
return results.map(({ id, isHibernated }) => [id, isHibernated]);
},
onChanged: async userIds => {
// We only update local copies since each process will get this event, but we can have user objects in multiple different caches.
// Before doing anything else we must "find" all the objects to update.
const userObjects = new Map<string, MiUser[]>();
const toUpdate: string[] = [];
for (const uid of userIds) {
const toAdd: MiUser[] = [];
const localUserById = this.localUserByIdCache.get(uid);
if (localUserById) toAdd.push(localUserById);
const userById = this.userByIdCache.get(uid);
if (userById) toAdd.push(userById);
if (toAdd.length > 0) {
toUpdate.push(uid);
userObjects.set(uid, toAdd);
}
}
// In many cases, we won't have to do anything.
// Skipping the DB fetch ensures that this remains a single-step synchronous process.
if (toUpdate.length > 0) {
const hibernations = await this.usersRepository.find({ where: { id: In(toUpdate) }, select: { id: true, isHibernated: true } });
for (const { id, isHibernated } of hibernations) {
const users = userObjects.get(id);
if (users) {
for (const u of users) {
u.isHibernated = isHibernated;
}
}
}
}
},
}); });
this.translationsCache = new RedisKVCache<CachedTranslationEntity>(this.redisClient, 'translations', { this.translationsCache = new RedisKVCache<CachedTranslationEntity>(this.redisClient, 'translations', {
@ -143,20 +238,21 @@ export class CacheService implements OnApplicationShutdown {
// NOTE: チャンネルのフォロー状況キャッシュはChannelFollowingServiceで行っている // NOTE: チャンネルのフォロー状況キャッシュはChannelFollowingServiceで行っている
this.redisForSub.on('message', this.onMessage); this.internalEventService.on('userChangeSuspendedState', this.onUserEvent);
this.internalEventService.on('userChangeDeletedState', this.onUserEvent);
this.internalEventService.on('remoteUserUpdated', this.onUserEvent);
this.internalEventService.on('localUserUpdated', this.onUserEvent);
this.internalEventService.on('userChangeSuspendedState', this.onUserEvent);
this.internalEventService.on('userTokenRegenerated', this.onTokenEvent);
this.internalEventService.on('follow', this.onFollowEvent);
this.internalEventService.on('unfollow', this.onFollowEvent);
} }
@bindThis @bindThis
private async onMessage(_: string, data: string): Promise<void> { private async onUserEvent<E extends 'userChangeSuspendedState' | 'userChangeDeletedState' | 'remoteUserUpdated' | 'localUserUpdated'>(body: InternalEventTypes[E], _: E, isLocal: boolean): Promise<void> {
const obj = JSON.parse(data); {
{
if (obj.channel === 'internal') { {
const { type, body } = obj.message as GlobalEvents['internal']['payload'];
switch (type) {
case 'userChangeSuspendedState':
case 'userChangeDeletedState':
case 'remoteUserUpdated':
case 'localUserUpdated': {
const user = await this.usersRepository.findOneBy({ id: body.id }); const user = await this.usersRepository.findOneBy({ id: body.id });
if (user == null) { if (user == null) {
this.userByIdCache.delete(body.id); this.userByIdCache.delete(body.id);
@ -166,6 +262,18 @@ export class CacheService implements OnApplicationShutdown {
this.uriPersonCache.delete(k); this.uriPersonCache.delete(k);
} }
} }
if (isLocal) {
await Promise.all([
this.userProfileCache.delete(body.id),
this.userMutingsCache.delete(body.id),
this.userBlockingCache.delete(body.id),
this.userBlockedCache.delete(body.id),
this.renoteMutingsCache.delete(body.id),
this.userFollowingsCache.delete(body.id),
this.userFollowersCache.delete(body.id),
this.hibernatedUserCache.delete(body.id),
]);
}
} else { } else {
this.userByIdCache.set(user.id, user); this.userByIdCache.set(user.id, user);
for (const [k, v] of this.uriPersonCache.entries) { for (const [k, v] of this.uriPersonCache.entries) {
@ -178,20 +286,37 @@ export class CacheService implements OnApplicationShutdown {
this.localUserByIdCache.set(user.id, user); this.localUserByIdCache.set(user.id, user);
} }
} }
break;
} }
case 'userTokenRegenerated': { }
}
}
@bindThis
private async onTokenEvent<E extends 'userTokenRegenerated'>(body: InternalEventTypes[E]): Promise<void> {
{
{
{
const user = await this.usersRepository.findOneByOrFail({ id: body.id }) as MiLocalUser; const user = await this.usersRepository.findOneByOrFail({ id: body.id }) as MiLocalUser;
this.localUserByNativeTokenCache.delete(body.oldToken); this.localUserByNativeTokenCache.delete(body.oldToken);
this.localUserByNativeTokenCache.set(body.newToken, user); this.localUserByNativeTokenCache.set(body.newToken, user);
break;
} }
}
}
}
@bindThis
private async onFollowEvent<E extends 'follow' | 'unfollow'>(body: InternalEventTypes[E], type: E): Promise<void> {
{
switch (type) {
case 'follow': { case 'follow': {
const follower = this.userByIdCache.get(body.followerId); const follower = this.userByIdCache.get(body.followerId);
if (follower) follower.followingCount++; if (follower) follower.followingCount++;
const followee = this.userByIdCache.get(body.followeeId); const followee = this.userByIdCache.get(body.followeeId);
if (followee) followee.followersCount++; if (followee) followee.followersCount++;
this.userFollowingsCache.delete(body.followerId); await Promise.all([
this.userFollowingsCache.delete(body.followerId),
this.userFollowersCache.delete(body.followeeId),
]);
this.userFollowStatsCache.delete(body.followerId); this.userFollowStatsCache.delete(body.followerId);
this.userFollowStatsCache.delete(body.followeeId); this.userFollowStatsCache.delete(body.followeeId);
break; break;
@ -201,13 +326,14 @@ export class CacheService implements OnApplicationShutdown {
if (follower) follower.followingCount--; if (follower) follower.followingCount--;
const followee = this.userByIdCache.get(body.followeeId); const followee = this.userByIdCache.get(body.followeeId);
if (followee) followee.followersCount--; if (followee) followee.followersCount--;
this.userFollowingsCache.delete(body.followerId); await Promise.all([
this.userFollowingsCache.delete(body.followerId),
this.userFollowersCache.delete(body.followeeId),
]);
this.userFollowStatsCache.delete(body.followerId); this.userFollowStatsCache.delete(body.followerId);
this.userFollowStatsCache.delete(body.followeeId); this.userFollowStatsCache.delete(body.followeeId);
break; break;
} }
default:
break;
} }
} }
} }
@ -298,9 +424,115 @@ export class CacheService implements OnApplicationShutdown {
}); });
} }
@bindThis
public async getUsers(userIds: Iterable<string>): Promise<Map<string, MiUser>> {
const users = new Map<string, MiUser>;
const toFetch: string[] = [];
for (const userId of userIds) {
const fromCache = this.userByIdCache.get(userId);
if (fromCache) {
users.set(userId, fromCache);
} else {
toFetch.push(userId);
}
}
if (toFetch.length > 0) {
const fetched = await this.usersRepository.findBy({
id: In(toFetch),
});
for (const user of fetched) {
users.set(user.id, user);
this.userByIdCache.set(user.id, user);
}
}
return users;
}
@bindThis
public async isFollowing(follower: string | { id: string }, followee: string | { id: string }): Promise<boolean> {
const followerId = typeof(follower) === 'string' ? follower : follower.id;
const followeeId = typeof(followee) === 'string' ? followee : followee.id;
// This lets us use whichever one is in memory, falling back to DB fetch via userFollowingsCache.
return this.userFollowersCache.get(followeeId)?.has(followerId)
?? (await this.userFollowingsCache.fetch(followerId)).has(followeeId);
}
/**
* Returns all hibernated followers.
*/
@bindThis
public async getHibernatedFollowers(followeeId: string): Promise<MiFollowing[]> {
const followers = await this.getFollowersWithHibernation(followeeId);
return followers.filter(f => f.isFollowerHibernated);
}
/**
* Returns all non-hibernated followers.
*/
@bindThis
public async getNonHibernatedFollowers(followeeId: string): Promise<MiFollowing[]> {
const followers = await this.getFollowersWithHibernation(followeeId);
return followers.filter(f => !f.isFollowerHibernated);
}
/**
* Returns follower relations with populated isFollowerHibernated.
* If you don't need this field, then please use userFollowersCache directly for reduced overhead.
*/
@bindThis
public async getFollowersWithHibernation(followeeId: string): Promise<MiFollowing[]> {
const followers = await this.userFollowersCache.fetch(followeeId);
const hibernations = await this.hibernatedUserCache.fetchMany(followers.keys()).then(fs => fs.reduce((map, f) => {
map.set(f[0], f[1]);
return map;
}, new Map<string, boolean>));
return Array.from(followers.values()).map(following => ({
...following,
isFollowerHibernated: hibernations.get(following.followerId) ?? false,
}));
}
/**
* Refreshes follower and following relations for the given user.
*/
@bindThis
public async refreshFollowRelationsFor(userId: string): Promise<void> {
const followings = await this.userFollowingsCache.refresh(userId);
const followees = Array.from(followings.values()).map(f => f.followeeId);
await this.userFollowersCache.deleteMany(followees);
}
@bindThis
public clear(): void {
this.userByIdCache.clear();
this.localUserByNativeTokenCache.clear();
this.localUserByIdCache.clear();
this.uriPersonCache.clear();
this.userProfileCache.clear();
this.userMutingsCache.clear();
this.userBlockingCache.clear();
this.userBlockedCache.clear();
this.renoteMutingsCache.clear();
this.userFollowingsCache.clear();
this.userFollowStatsCache.clear();
this.translationsCache.clear();
}
@bindThis @bindThis
public dispose(): void { public dispose(): void {
this.redisForSub.off('message', this.onMessage); this.internalEventService.off('userChangeSuspendedState', this.onUserEvent);
this.internalEventService.off('userChangeDeletedState', this.onUserEvent);
this.internalEventService.off('remoteUserUpdated', this.onUserEvent);
this.internalEventService.off('localUserUpdated', this.onUserEvent);
this.internalEventService.off('userChangeSuspendedState', this.onUserEvent);
this.internalEventService.off('userTokenRegenerated', this.onTokenEvent);
this.internalEventService.off('follow', this.onFollowEvent);
this.internalEventService.off('unfollow', this.onFollowEvent);
this.userByIdCache.dispose(); this.userByIdCache.dispose();
this.localUserByNativeTokenCache.dispose(); this.localUserByNativeTokenCache.dispose();
this.localUserByIdCache.dispose(); this.localUserByIdCache.dispose();

View file

@ -9,14 +9,15 @@ import { DI } from '@/di-symbols.js';
import type { ChannelFollowingsRepository } from '@/models/_.js'; import type { ChannelFollowingsRepository } from '@/models/_.js';
import { MiChannel } from '@/models/_.js'; import { MiChannel } from '@/models/_.js';
import { IdService } from '@/core/IdService.js'; import { IdService } from '@/core/IdService.js';
import { GlobalEvents, GlobalEventService } from '@/core/GlobalEventService.js'; import { GlobalEvents, GlobalEventService, InternalEventTypes } from '@/core/GlobalEventService.js';
import { bindThis } from '@/decorators.js'; import { bindThis } from '@/decorators.js';
import type { MiLocalUser } from '@/models/User.js'; import type { MiLocalUser } from '@/models/User.js';
import { RedisKVCache } from '@/misc/cache.js'; import { QuantumKVCache } from '@/misc/QuantumKVCache.js';
import { InternalEventService } from './InternalEventService.js';
@Injectable() @Injectable()
export class ChannelFollowingService implements OnModuleInit { export class ChannelFollowingService implements OnModuleInit {
public userFollowingChannelsCache: RedisKVCache<Set<string>>; public userFollowingChannelsCache: QuantumKVCache<Set<string>>;
constructor( constructor(
@Inject(DI.redis) @Inject(DI.redis)
@ -27,19 +28,18 @@ export class ChannelFollowingService implements OnModuleInit {
private channelFollowingsRepository: ChannelFollowingsRepository, private channelFollowingsRepository: ChannelFollowingsRepository,
private idService: IdService, private idService: IdService,
private globalEventService: GlobalEventService, private globalEventService: GlobalEventService,
private readonly internalEventService: InternalEventService,
) { ) {
this.userFollowingChannelsCache = new RedisKVCache<Set<string>>(this.redisClient, 'userFollowingChannels', { this.userFollowingChannelsCache = new QuantumKVCache<Set<string>>(this.internalEventService, 'userFollowingChannels', {
lifetime: 1000 * 60 * 30, // 30m lifetime: 1000 * 60 * 30, // 30m
memoryCacheLifetime: 1000 * 60, // 1m
fetcher: (key) => this.channelFollowingsRepository.find({ fetcher: (key) => this.channelFollowingsRepository.find({
where: { followerId: key }, where: { followerId: key },
select: ['followeeId'], select: ['followeeId'],
}).then(xs => new Set(xs.map(x => x.followeeId))), }).then(xs => new Set(xs.map(x => x.followeeId))),
toRedisConverter: (value) => JSON.stringify(Array.from(value)),
fromRedisConverter: (value) => new Set(JSON.parse(value)),
}); });
this.redisForSub.on('message', this.onMessage); this.internalEventService.on('followChannel', this.onMessage);
this.internalEventService.on('unfollowChannel', this.onMessage);
} }
onModuleInit() { onModuleInit() {
@ -79,18 +79,15 @@ export class ChannelFollowingService implements OnModuleInit {
} }
@bindThis @bindThis
private async onMessage(_: string, data: string): Promise<void> { private async onMessage<E extends 'followChannel' | 'unfollowChannel'>(body: InternalEventTypes[E], type: E): Promise<void> {
const obj = JSON.parse(data); {
if (obj.channel === 'internal') {
const { type, body } = obj.message as GlobalEvents['internal']['payload'];
switch (type) { switch (type) {
case 'followChannel': { case 'followChannel': {
this.userFollowingChannelsCache.refresh(body.userId); await this.userFollowingChannelsCache.delete(body.userId);
break; break;
} }
case 'unfollowChannel': { case 'unfollowChannel': {
this.userFollowingChannelsCache.delete(body.userId); await this.userFollowingChannelsCache.delete(body.userId);
break; break;
} }
} }
@ -99,6 +96,8 @@ export class ChannelFollowingService implements OnModuleInit {
@bindThis @bindThis
public dispose(): void { public dispose(): void {
this.internalEventService.off('followChannel', this.onMessage);
this.internalEventService.off('unfollowChannel', this.onMessage);
this.userFollowingChannelsCache.dispose(); this.userFollowingChannelsCache.dispose();
} }

View file

@ -41,6 +41,7 @@ import { HttpRequestService } from './HttpRequestService.js';
import { IdService } from './IdService.js'; import { IdService } from './IdService.js';
import { ImageProcessingService } from './ImageProcessingService.js'; import { ImageProcessingService } from './ImageProcessingService.js';
import { SystemAccountService } from './SystemAccountService.js'; import { SystemAccountService } from './SystemAccountService.js';
import { InternalEventService } from './InternalEventService.js';
import { InternalStorageService } from './InternalStorageService.js'; import { InternalStorageService } from './InternalStorageService.js';
import { MetaService } from './MetaService.js'; import { MetaService } from './MetaService.js';
import { MfmService } from './MfmService.js'; import { MfmService } from './MfmService.js';
@ -186,6 +187,7 @@ const $HashtagService: Provider = { provide: 'HashtagService', useExisting: Hash
const $HttpRequestService: Provider = { provide: 'HttpRequestService', useExisting: HttpRequestService }; const $HttpRequestService: Provider = { provide: 'HttpRequestService', useExisting: HttpRequestService };
const $IdService: Provider = { provide: 'IdService', useExisting: IdService }; const $IdService: Provider = { provide: 'IdService', useExisting: IdService };
const $ImageProcessingService: Provider = { provide: 'ImageProcessingService', useExisting: ImageProcessingService }; const $ImageProcessingService: Provider = { provide: 'ImageProcessingService', useExisting: ImageProcessingService };
const $InternalEventService: Provider = { provide: 'InternalEventService', useExisting: InternalEventService };
const $InternalStorageService: Provider = { provide: 'InternalStorageService', useExisting: InternalStorageService }; const $InternalStorageService: Provider = { provide: 'InternalStorageService', useExisting: InternalStorageService };
const $MetaService: Provider = { provide: 'MetaService', useExisting: MetaService }; const $MetaService: Provider = { provide: 'MetaService', useExisting: MetaService };
const $MfmService: Provider = { provide: 'MfmService', useExisting: MfmService }; const $MfmService: Provider = { provide: 'MfmService', useExisting: MfmService };
@ -345,6 +347,7 @@ const $SponsorsService: Provider = { provide: 'SponsorsService', useExisting: Sp
HttpRequestService, HttpRequestService,
IdService, IdService,
ImageProcessingService, ImageProcessingService,
InternalEventService,
InternalStorageService, InternalStorageService,
MetaService, MetaService,
MfmService, MfmService,
@ -500,6 +503,7 @@ const $SponsorsService: Provider = { provide: 'SponsorsService', useExisting: Sp
$HttpRequestService, $HttpRequestService,
$IdService, $IdService,
$ImageProcessingService, $ImageProcessingService,
$InternalEventService,
$InternalStorageService, $InternalStorageService,
$MetaService, $MetaService,
$MfmService, $MfmService,
@ -656,6 +660,7 @@ const $SponsorsService: Provider = { provide: 'SponsorsService', useExisting: Sp
HttpRequestService, HttpRequestService,
IdService, IdService,
ImageProcessingService, ImageProcessingService,
InternalEventService,
InternalStorageService, InternalStorageService,
MetaService, MetaService,
MfmService, MfmService,
@ -810,6 +815,7 @@ const $SponsorsService: Provider = { provide: 'SponsorsService', useExisting: Sp
$HttpRequestService, $HttpRequestService,
$IdService, $IdService,
$ImageProcessingService, $ImageProcessingService,
$InternalEventService,
$InternalStorageService, $InternalStorageService,
$MetaService, $MetaService,
$MfmService, $MfmService,

View file

@ -265,6 +265,7 @@ export interface InternalEventTypes {
unmute: { muterId: MiUser['id']; muteeId: MiUser['id']; }; unmute: { muterId: MiUser['id']; muteeId: MiUser['id']; };
userListMemberAdded: { userListId: MiUserList['id']; memberId: MiUser['id']; }; userListMemberAdded: { userListId: MiUserList['id']; memberId: MiUser['id']; };
userListMemberRemoved: { userListId: MiUserList['id']; memberId: MiUser['id']; }; userListMemberRemoved: { userListId: MiUserList['id']; memberId: MiUser['id']; };
quantumCacheUpdated: { name: string, keys: string[] };
} }
type EventTypesToEventPayload<T> = EventUnionFromDictionary<UndefinedAsNullAll<SerializedAll<T>>>; type EventTypesToEventPayload<T> = EventUnionFromDictionary<UndefinedAsNullAll<SerializedAll<T>>>;
@ -353,12 +354,12 @@ export class GlobalEventService {
} }
@bindThis @bindThis
private publish(channel: StreamChannels, type: string | null, value?: any): void { private async publish(channel: StreamChannels, type: string | null, value?: any): Promise<void> {
const message = type == null ? value : value == null ? const message = type == null ? value : value == null ?
{ type: type, body: null } : { type: type, body: null } :
{ type: type, body: value }; { type: type, body: value };
this.redisForPub.publish(this.config.host, JSON.stringify({ await this.redisForPub.publish(this.config.host, JSON.stringify({
channel: channel, channel: channel,
message: message, message: message,
})); }));
@ -369,6 +370,11 @@ export class GlobalEventService {
this.publish('internal', type, typeof value === 'undefined' ? null : value); this.publish('internal', type, typeof value === 'undefined' ? null : value);
} }
@bindThis
public async publishInternalEventAsync<K extends keyof InternalEventTypes>(type: K, value?: InternalEventTypes[K]): Promise<void> {
await this.publish('internal', type, typeof value === 'undefined' ? null : value);
}
@bindThis @bindThis
public publishBroadcastStream<K extends keyof BroadcastTypes>(type: K, value?: BroadcastTypes[K]): void { public publishBroadcastStream<K extends keyof BroadcastTypes>(type: K, value?: BroadcastTypes[K]): void {
this.publish('broadcast', type, typeof value === 'undefined' ? null : value); this.publish('broadcast', type, typeof value === 'undefined' ? null : value);

View file

@ -0,0 +1,103 @@
/*
* SPDX-FileCopyrightText: hazelnoot and other Sharkey contributors
* SPDX-License-Identifier: AGPL-3.0-only
*/
import { Inject, Injectable, OnApplicationShutdown } from '@nestjs/common';
import Redis from 'ioredis';
import { DI } from '@/di-symbols.js';
import { GlobalEventService } from '@/core/GlobalEventService.js';
import type { GlobalEvents, InternalEventTypes } from '@/core/GlobalEventService.js';
import { bindThis } from '@/decorators.js';
export type Listener<K extends keyof InternalEventTypes> = (value: InternalEventTypes[K], key: K, isLocal: boolean) => void | Promise<void>;
export interface ListenerProps {
ignoreLocal?: boolean,
ignoreRemote?: boolean,
}
@Injectable()
export class InternalEventService implements OnApplicationShutdown {
private readonly listeners = new Map<keyof InternalEventTypes, Map<Listener<keyof InternalEventTypes>, ListenerProps>>();
constructor(
@Inject(DI.redisForSub)
private readonly redisForSub: Redis.Redis,
private readonly globalEventService: GlobalEventService,
) {
this.redisForSub.on('message', this.onMessage);
}
@bindThis
public on<K extends keyof InternalEventTypes>(type: K, listener: Listener<K>, props?: ListenerProps): void {
let set = this.listeners.get(type);
if (!set) {
set = new Map();
this.listeners.set(type, set);
}
// Functionally, this is just a set with metadata on the values.
set.set(listener as Listener<keyof InternalEventTypes>, props ?? {});
}
@bindThis
public off<K extends keyof InternalEventTypes>(type: K, listener: Listener<K>): void {
this.listeners.get(type)?.delete(listener as Listener<keyof InternalEventTypes>);
}
@bindThis
public async emit<K extends keyof InternalEventTypes>(type: K, value: InternalEventTypes[K]): Promise<void> {
await this.emitInternal(type, value, true);
await this.globalEventService.publishInternalEventAsync(type, { ...value, _pid: process.pid });
}
@bindThis
private async emitInternal<K extends keyof InternalEventTypes>(type: K, value: InternalEventTypes[K], isLocal: boolean): Promise<void> {
const listeners = this.listeners.get(type);
if (!listeners) {
return;
}
const promises: Promise<void>[] = [];
for (const [listener, props] of listeners) {
if ((isLocal && !props.ignoreLocal) || (!isLocal && !props.ignoreRemote)) {
const promise = Promise.resolve(listener(value, type, isLocal));
promises.push(promise);
}
}
await Promise.all(promises);
}
@bindThis
private async onMessage(_: string, data: string): Promise<void> {
const obj = JSON.parse(data);
if (obj.channel === 'internal') {
const { type, body } = obj.message as GlobalEvents['internal']['payload'];
if (!isLocalInternalEvent(body) || body._pid !== process.pid) {
await this.emitInternal(type, body as InternalEventTypes[keyof InternalEventTypes], false);
}
}
}
@bindThis
public dispose(): void {
this.redisForSub.off('message', this.onMessage);
this.listeners.clear();
}
@bindThis
public onApplicationShutdown(): void {
this.dispose();
}
}
interface LocalInternalEvent {
_pid: number;
}
function isLocalInternalEvent(body: object): body is LocalInternalEvent {
return '_pid' in body && typeof(body._pid) === 'number';
}

View file

@ -606,11 +606,11 @@ export class NoteCreateService implements OnApplicationShutdown {
} }
if (data.reply == null) { if (data.reply == null) {
// TODO: キャッシュ this.cacheService.userFollowersCache.fetch(user.id).then(async followingsMap => {
this.followingsRepository.findBy({ const followings = Array
followeeId: user.id, .from(followingsMap.values())
notify: 'normal', .filter(f => f.notify === 'normal');
}).then(async followings => {
if (note.visibility !== 'specified') { if (note.visibility !== 'specified') {
const isPureRenote = this.isRenote(data) && !this.isQuote(data) ? true : false; const isPureRenote = this.isRenote(data) && !this.isQuote(data) ? true : false;
for (const following of followings) { for (const following of followings) {
@ -948,14 +948,7 @@ export class NoteCreateService implements OnApplicationShutdown {
// TODO: キャッシュ? // TODO: キャッシュ?
// eslint-disable-next-line prefer-const // eslint-disable-next-line prefer-const
let [followings, userListMemberships] = await Promise.all([ let [followings, userListMemberships] = await Promise.all([
this.followingsRepository.find({ this.cacheService.getNonHibernatedFollowers(user.id),
where: {
followeeId: user.id,
followerHost: IsNull(),
isFollowerHibernated: false,
},
select: ['followerId', 'withReplies'],
}),
this.userListMembershipsRepository.find({ this.userListMembershipsRepository.find({
where: { where: {
userId: user.id, userId: user.id,
@ -1072,17 +1065,19 @@ export class NoteCreateService implements OnApplicationShutdown {
}); });
if (hibernatedUsers.length > 0) { if (hibernatedUsers.length > 0) {
await Promise.all([
this.usersRepository.update({ this.usersRepository.update({
id: In(hibernatedUsers.map(x => x.id)), id: In(hibernatedUsers.map(x => x.id)),
}, { }, {
isHibernated: true, isHibernated: true,
}); }),
this.followingsRepository.update({ this.followingsRepository.update({
followerId: In(hibernatedUsers.map(x => x.id)), followerId: In(hibernatedUsers.map(x => x.id)),
}, { }, {
isFollowerHibernated: true, isFollowerHibernated: true,
}); }),
this.cacheService.hibernatedUserCache.setMany(hibernatedUsers.map(x => [x.id, true])),
]);
} }
} }

View file

@ -833,14 +833,7 @@ export class NoteEditService implements OnApplicationShutdown {
// TODO: キャッシュ? // TODO: キャッシュ?
// eslint-disable-next-line prefer-const // eslint-disable-next-line prefer-const
let [followings, userListMemberships] = await Promise.all([ let [followings, userListMemberships] = await Promise.all([
this.followingsRepository.find({ this.cacheService.getNonHibernatedFollowers(user.id),
where: {
followeeId: user.id,
followerHost: IsNull(),
isFollowerHibernated: false,
},
select: ['followerId', 'withReplies'],
}),
this.userListMembershipsRepository.find({ this.userListMembershipsRepository.find({
where: { where: {
userId: user.id, userId: user.id,
@ -957,17 +950,19 @@ export class NoteEditService implements OnApplicationShutdown {
}); });
if (hibernatedUsers.length > 0) { if (hibernatedUsers.length > 0) {
await Promise.all([
this.usersRepository.update({ this.usersRepository.update({
id: In(hibernatedUsers.map(x => x.id)), id: In(hibernatedUsers.map(x => x.id)),
}, { }, {
isHibernated: true, isHibernated: true,
}); }),
this.followingsRepository.update({ this.followingsRepository.update({
followerId: In(hibernatedUsers.map(x => x.id)), followerId: In(hibernatedUsers.map(x => x.id)),
}, { }, {
isFollowerHibernated: true, isFollowerHibernated: true,
}); }),
this.cacheService.hibernatedUserCache.setMany(hibernatedUsers.map(x => [x.id, true])),
]);
} }
} }

View file

@ -113,27 +113,27 @@ export class NotificationService implements OnApplicationShutdown {
} }
if (recieveConfig?.type === 'following') { if (recieveConfig?.type === 'following') {
const isFollowing = await this.cacheService.userFollowingsCache.fetch(notifieeId).then(followings => Object.hasOwn(followings, notifierId)); const isFollowing = await this.cacheService.userFollowingsCache.fetch(notifieeId).then(followings => followings.has(notifierId));
if (!isFollowing) { if (!isFollowing) {
return null; return null;
} }
} else if (recieveConfig?.type === 'follower') { } else if (recieveConfig?.type === 'follower') {
const isFollower = await this.cacheService.userFollowingsCache.fetch(notifierId).then(followings => Object.hasOwn(followings, notifieeId)); const isFollower = await this.cacheService.userFollowingsCache.fetch(notifierId).then(followings => followings.has(notifieeId));
if (!isFollower) { if (!isFollower) {
return null; return null;
} }
} else if (recieveConfig?.type === 'mutualFollow') { } else if (recieveConfig?.type === 'mutualFollow') {
const [isFollowing, isFollower] = await Promise.all([ const [isFollowing, isFollower] = await Promise.all([
this.cacheService.userFollowingsCache.fetch(notifieeId).then(followings => Object.hasOwn(followings, notifierId)), this.cacheService.userFollowingsCache.fetch(notifieeId).then(followings => followings.has(notifierId)),
this.cacheService.userFollowingsCache.fetch(notifierId).then(followings => Object.hasOwn(followings, notifieeId)), this.cacheService.userFollowingsCache.fetch(notifierId).then(followings => followings.has(notifieeId)),
]); ]);
if (!(isFollowing && isFollower)) { if (!(isFollowing && isFollower)) {
return null; return null;
} }
} else if (recieveConfig?.type === 'followingOrFollower') { } else if (recieveConfig?.type === 'followingOrFollower') {
const [isFollowing, isFollower] = await Promise.all([ const [isFollowing, isFollower] = await Promise.all([
this.cacheService.userFollowingsCache.fetch(notifieeId).then(followings => Object.hasOwn(followings, notifierId)), this.cacheService.userFollowingsCache.fetch(notifieeId).then(followings => followings.has(notifierId)),
this.cacheService.userFollowingsCache.fetch(notifierId).then(followings => Object.hasOwn(followings, notifieeId)), this.cacheService.userFollowingsCache.fetch(notifierId).then(followings => followings.has(notifieeId)),
]); ]);
if (!isFollowing && !isFollower) { if (!isFollowing && !isFollower) {
return null; return null;

View file

@ -12,7 +12,8 @@ import type { Packed } from '@/misc/json-schema.js';
import { getNoteSummary } from '@/misc/get-note-summary.js'; import { getNoteSummary } from '@/misc/get-note-summary.js';
import type { MiMeta, MiSwSubscription, SwSubscriptionsRepository } from '@/models/_.js'; import type { MiMeta, MiSwSubscription, SwSubscriptionsRepository } from '@/models/_.js';
import { bindThis } from '@/decorators.js'; import { bindThis } from '@/decorators.js';
import { RedisKVCache } from '@/misc/cache.js'; import { QuantumKVCache } from '@/misc/QuantumKVCache.js';
import { InternalEventService } from '@/core/InternalEventService.js';
// Defined also packages/sw/types.ts#L13 // Defined also packages/sw/types.ts#L13
type PushNotificationsTypes = { type PushNotificationsTypes = {
@ -48,7 +49,7 @@ function truncateBody<T extends keyof PushNotificationsTypes>(type: T, body: Pus
@Injectable() @Injectable()
export class PushNotificationService implements OnApplicationShutdown { export class PushNotificationService implements OnApplicationShutdown {
private subscriptionsCache: RedisKVCache<MiSwSubscription[]>; private subscriptionsCache: QuantumKVCache<MiSwSubscription[]>;
constructor( constructor(
@Inject(DI.config) @Inject(DI.config)
@ -62,13 +63,11 @@ export class PushNotificationService implements OnApplicationShutdown {
@Inject(DI.swSubscriptionsRepository) @Inject(DI.swSubscriptionsRepository)
private swSubscriptionsRepository: SwSubscriptionsRepository, private swSubscriptionsRepository: SwSubscriptionsRepository,
private readonly internalEventService: InternalEventService,
) { ) {
this.subscriptionsCache = new RedisKVCache<MiSwSubscription[]>(this.redisClient, 'userSwSubscriptions', { this.subscriptionsCache = new QuantumKVCache<MiSwSubscription[]>(this.internalEventService, 'userSwSubscriptions', {
lifetime: 1000 * 60 * 60 * 1, // 1h lifetime: 1000 * 60 * 60 * 1, // 1h
memoryCacheLifetime: 1000 * 60 * 3, // 3m
fetcher: (key) => this.swSubscriptionsRepository.findBy({ userId: key }), fetcher: (key) => this.swSubscriptionsRepository.findBy({ userId: key }),
toRedisConverter: (value) => JSON.stringify(value),
fromRedisConverter: (value) => JSON.parse(value),
}); });
} }
@ -114,8 +113,8 @@ export class PushNotificationService implements OnApplicationShutdown {
endpoint: subscription.endpoint, endpoint: subscription.endpoint,
auth: subscription.auth, auth: subscription.auth,
publickey: subscription.publickey, publickey: subscription.publickey,
}).then(() => { }).then(async () => {
this.refreshCache(userId); await this.refreshCache(userId);
}); });
} }
}); });
@ -123,8 +122,8 @@ export class PushNotificationService implements OnApplicationShutdown {
} }
@bindThis @bindThis
public refreshCache(userId: string): void { public async refreshCache(userId: string): Promise<void> {
this.subscriptionsCache.refresh(userId); await this.subscriptionsCache.refresh(userId);
} }
@bindThis @bindThis

View file

@ -94,7 +94,7 @@ export class QueryService {
@bindThis @bindThis
public generateBlockQueryForUsers<E extends ObjectLiteral>(q: SelectQueryBuilder<E>, me: { id: MiUser['id'] }): SelectQueryBuilder<E> { public generateBlockQueryForUsers<E extends ObjectLiteral>(q: SelectQueryBuilder<E>, me: { id: MiUser['id'] }): SelectQueryBuilder<E> {
this.andNotBlockingUser(q, ':meId', 'user.id'); this.andNotBlockingUser(q, ':meId', 'user.id');
this.andNotBlockingUser(q, 'user.id', ':me.id'); this.andNotBlockingUser(q, 'user.id', ':meId');
return q.setParameters({ meId: me.id }); return q.setParameters({ meId: me.id });
} }

View file

@ -122,7 +122,7 @@ export class ReactionService {
} }
// check visibility // check visibility
if (!await this.noteEntityService.isVisibleForMe(note, user.id)) { if (!await this.noteEntityService.isVisibleForMe(note, user.id, { me: user })) {
throw new IdentifiableError('68e9d2d1-48bf-42c2-b90a-b20e09fd3d48', 'Note not accessible for you.'); throw new IdentifiableError('68e9d2d1-48bf-42c2-b90a-b20e09fd3d48', 'Note not accessible for you.');
} }

View file

@ -77,8 +77,10 @@ export class UserBlockingService implements OnModuleInit {
await this.blockingsRepository.insert(blocking); await this.blockingsRepository.insert(blocking);
this.cacheService.userBlockingCache.refresh(blocker.id); await Promise.all([
this.cacheService.userBlockedCache.refresh(blockee.id); this.cacheService.userBlockingCache.delete(blocker.id),
this.cacheService.userBlockedCache.delete(blockee.id),
]);
this.globalEventService.publishInternalEvent('blockingCreated', { this.globalEventService.publishInternalEvent('blockingCreated', {
blockerId: blocker.id, blockerId: blocker.id,
@ -168,8 +170,10 @@ export class UserBlockingService implements OnModuleInit {
await this.blockingsRepository.delete(blocking.id); await this.blockingsRepository.delete(blocking.id);
this.cacheService.userBlockingCache.refresh(blocker.id); await Promise.all([
this.cacheService.userBlockedCache.refresh(blockee.id); this.cacheService.userBlockingCache.delete(blocker.id),
this.cacheService.userBlockedCache.delete(blockee.id),
]);
this.globalEventService.publishInternalEvent('blockingDeleted', { this.globalEventService.publishInternalEvent('blockingDeleted', {
blockerId: blocker.id, blockerId: blocker.id,

View file

@ -29,6 +29,7 @@ import { AccountMoveService } from '@/core/AccountMoveService.js';
import { UtilityService } from '@/core/UtilityService.js'; import { UtilityService } from '@/core/UtilityService.js';
import type { ThinUser } from '@/queue/types.js'; import type { ThinUser } from '@/queue/types.js';
import { LoggerService } from '@/core/LoggerService.js'; import { LoggerService } from '@/core/LoggerService.js';
import { InternalEventService } from '@/core/InternalEventService.js';
import type Logger from '../logger.js'; import type Logger from '../logger.js';
type Local = MiLocalUser | { type Local = MiLocalUser | {
@ -86,6 +87,7 @@ export class UserFollowingService implements OnModuleInit {
private accountMoveService: AccountMoveService, private accountMoveService: AccountMoveService,
private perUserFollowingChart: PerUserFollowingChart, private perUserFollowingChart: PerUserFollowingChart,
private instanceChart: InstanceChart, private instanceChart: InstanceChart,
private readonly internalEventService: InternalEventService,
loggerService: LoggerService, loggerService: LoggerService,
) { ) {
@ -145,12 +147,7 @@ export class UserFollowingService implements OnModuleInit {
if (blocked) throw new IdentifiableError('3338392a-f764-498d-8855-db939dcf8c48', 'blocked'); if (blocked) throw new IdentifiableError('3338392a-f764-498d-8855-db939dcf8c48', 'blocked');
} }
if (await this.followingsRepository.exists({ if (await this.cacheService.isFollowing(follower, followee)) {
where: {
followerId: follower.id,
followeeId: followee.id,
},
})) {
// すでにフォロー関係が存在している場合 // すでにフォロー関係が存在している場合
if (this.userEntityService.isRemoteUser(follower) && this.userEntityService.isLocalUser(followee)) { if (this.userEntityService.isRemoteUser(follower) && this.userEntityService.isLocalUser(followee)) {
// リモート → ローカル: acceptを送り返しておしまい // リモート → ローカル: acceptを送り返しておしまい
@ -178,24 +175,14 @@ export class UserFollowingService implements OnModuleInit {
let autoAccept = false; let autoAccept = false;
// 鍵アカウントであっても、既にフォローされていた場合はスルー // 鍵アカウントであっても、既にフォローされていた場合はスルー
const isFollowing = await this.followingsRepository.exists({ const isFollowing = await this.cacheService.isFollowing(follower, followee);
where: {
followerId: follower.id,
followeeId: followee.id,
},
});
if (isFollowing) { if (isFollowing) {
autoAccept = true; autoAccept = true;
} }
// フォローしているユーザーは自動承認オプション // フォローしているユーザーは自動承認オプション
if (!autoAccept && (this.userEntityService.isLocalUser(followee) && followeeProfile.autoAcceptFollowed)) { if (!autoAccept && (this.userEntityService.isLocalUser(followee) && followeeProfile.autoAcceptFollowed)) {
const isFollowed = await this.followingsRepository.exists({ const isFollowed = await this.cacheService.isFollowing(followee, follower); // intentionally reversed parameters
where: {
followerId: followee.id,
followeeId: follower.id,
},
});
if (isFollowed) autoAccept = true; if (isFollowed) autoAccept = true;
} }
@ -204,12 +191,7 @@ export class UserFollowingService implements OnModuleInit {
if (followee.isLocked && !autoAccept) { if (followee.isLocked && !autoAccept) {
autoAccept = !!(await this.accountMoveService.validateAlsoKnownAs( autoAccept = !!(await this.accountMoveService.validateAlsoKnownAs(
follower, follower,
(oldSrc, newSrc) => this.followingsRepository.exists({ (oldSrc, newSrc) => this.cacheService.isFollowing(newSrc, followee),
where: {
followeeId: followee.id,
followerId: newSrc.id,
},
}),
true, true,
)); ));
} }
@ -264,7 +246,8 @@ export class UserFollowingService implements OnModuleInit {
} }
}); });
this.cacheService.userFollowingsCache.refresh(follower.id); // Handled by CacheService
//this.cacheService.userFollowingsCache.refresh(follower.id);
const requestExist = await this.followRequestsRepository.exists({ const requestExist = await this.followRequestsRepository.exists({
where: { where: {
@ -291,7 +274,7 @@ export class UserFollowingService implements OnModuleInit {
}, followee.id); }, followee.id);
} }
this.globalEventService.publishInternalEvent('follow', { followerId: follower.id, followeeId: followee.id }); await this.internalEventService.emit('follow', { followerId: follower.id, followeeId: followee.id });
const [followeeUser, followerUser] = await Promise.all([ const [followeeUser, followerUser] = await Promise.all([
this.usersRepository.findOneByOrFail({ id: followee.id }), this.usersRepository.findOneByOrFail({ id: followee.id }),
@ -363,31 +346,29 @@ export class UserFollowingService implements OnModuleInit {
}, },
silent = false, silent = false,
): Promise<void> { ): Promise<void> {
const following = await this.followingsRepository.findOne({ const [
relations: { followerUser,
follower: true, followeeUser,
followee: true, following,
}, ] = await Promise.all([
where: { this.cacheService.findUserById(follower.id),
followerId: follower.id, this.cacheService.findUserById(followee.id),
followeeId: followee.id, this.cacheService.userFollowingsCache.fetch(follower.id).then(fs => fs.get(followee.id)),
}, ]);
});
if (following === null || !following.follower || !following.followee) { if (following == null) {
this.logger.warn('フォロー解除がリクエストされましたがフォローしていませんでした'); this.logger.warn('フォロー解除がリクエストされましたがフォローしていませんでした');
return; return;
} }
await this.followingsRepository.delete(following.id); await this.followingsRepository.delete(following.id);
await this.internalEventService.emit('unfollow', { followerId: follower.id, followeeId: followee.id });
this.cacheService.userFollowingsCache.refresh(follower.id); this.decrementFollowing(followerUser, followeeUser);
this.decrementFollowing(following.follower, following.followee);
if (!silent && this.userEntityService.isLocalUser(follower)) { if (!silent && this.userEntityService.isLocalUser(follower)) {
// Publish unfollow event // Publish unfollow event
this.userEntityService.pack(followee.id, follower, { this.userEntityService.pack(followeeUser, follower, {
schema: 'UserDetailedNotMe', schema: 'UserDetailedNotMe',
}).then(async packed => { }).then(async packed => {
this.globalEventService.publishMainStream(follower.id, 'unfollow', packed); this.globalEventService.publishMainStream(follower.id, 'unfollow', packed);
@ -412,8 +393,6 @@ export class UserFollowingService implements OnModuleInit {
follower: MiUser, follower: MiUser,
followee: MiUser, followee: MiUser,
): Promise<void> { ): Promise<void> {
this.globalEventService.publishInternalEvent('unfollow', { followerId: follower.id, followeeId: followee.id });
// Neither followee nor follower has moved. // Neither followee nor follower has moved.
if (!follower.movedToUri && !followee.movedToUri) { if (!follower.movedToUri && !followee.movedToUri) {
//#region Decrement following / followers counts //#region Decrement following / followers counts
@ -687,22 +666,22 @@ export class UserFollowingService implements OnModuleInit {
*/ */
@bindThis @bindThis
private async removeFollow(followee: Both, follower: Both): Promise<void> { private async removeFollow(followee: Both, follower: Both): Promise<void> {
const following = await this.followingsRepository.findOne({ const [
relations: { followerUser,
followee: true, followeeUser,
follower: true, following,
}, ] = await Promise.all([
where: { this.cacheService.findUserById(follower.id),
followeeId: followee.id, this.cacheService.findUserById(followee.id),
followerId: follower.id, this.cacheService.userFollowingsCache.fetch(follower.id).then(fs => fs.get(followee.id)),
}, ]);
});
if (!following || !following.followee || !following.follower) return; if (!following) return;
await this.followingsRepository.delete(following.id); await this.followingsRepository.delete(following.id);
await this.internalEventService.emit('unfollow', { followerId: follower.id, followeeId: followee.id });
this.decrementFollowing(following.follower, following.followee); this.decrementFollowing(followerUser, followeeUser);
} }
/** /**
@ -733,36 +712,26 @@ export class UserFollowingService implements OnModuleInit {
} }
@bindThis @bindThis
public getFollowees(userId: MiUser['id']) { public async getFollowees(userId: MiUser['id']) {
return this.followingsRepository.createQueryBuilder('following') const followings = await this.cacheService.userFollowingsCache.fetch(userId);
.select('following.followeeId') return Array.from(followings.values());
.where('following.followerId = :followerId', { followerId: userId })
.getMany();
} }
@bindThis @bindThis
public isFollowing(followerId: MiUser['id'], followeeId: MiUser['id']) { public async isFollowing(followerId: MiUser['id'], followeeId: MiUser['id']) {
return this.followingsRepository.exists({ return this.cacheService.isFollowing(followerId, followeeId);
where: {
followerId,
followeeId,
},
});
} }
@bindThis @bindThis
public async isMutual(aUserId: MiUser['id'], bUserId: MiUser['id']) { public async isMutual(aUserId: MiUser['id'], bUserId: MiUser['id']) {
const count = await this.followingsRepository.createQueryBuilder('following') const [
.where(new Brackets(qb => { isFollowing,
qb.where('following.followerId = :aUserId', { aUserId }) isFollowed,
.andWhere('following.followeeId = :bUserId', { bUserId }); ] = await Promise.all([
})) this.isFollowing(aUserId, bUserId),
.orWhere(new Brackets(qb => { this.isFollowing(bUserId, aUserId),
qb.where('following.followerId = :bUserId', { bUserId }) ]);
.andWhere('following.followeeId = :aUserId', { aUserId });
}))
.getCount();
return count === 2; return isFollowing && isFollowed;
} }
} }

View file

@ -7,14 +7,14 @@ import { Inject, Injectable, OnApplicationShutdown } from '@nestjs/common';
import * as Redis from 'ioredis'; import * as Redis from 'ioredis';
import type { MiUser } from '@/models/User.js'; import type { MiUser } from '@/models/User.js';
import type { UserKeypairsRepository } from '@/models/_.js'; import type { UserKeypairsRepository } from '@/models/_.js';
import { RedisKVCache } from '@/misc/cache.js'; import { MemoryKVCache, RedisKVCache } from '@/misc/cache.js';
import type { MiUserKeypair } from '@/models/UserKeypair.js'; import type { MiUserKeypair } from '@/models/UserKeypair.js';
import { DI } from '@/di-symbols.js'; import { DI } from '@/di-symbols.js';
import { bindThis } from '@/decorators.js'; import { bindThis } from '@/decorators.js';
@Injectable() @Injectable()
export class UserKeypairService implements OnApplicationShutdown { export class UserKeypairService implements OnApplicationShutdown {
private cache: RedisKVCache<MiUserKeypair>; private cache: MemoryKVCache<MiUserKeypair>;
constructor( constructor(
@Inject(DI.redis) @Inject(DI.redis)
@ -23,18 +23,12 @@ export class UserKeypairService implements OnApplicationShutdown {
@Inject(DI.userKeypairsRepository) @Inject(DI.userKeypairsRepository)
private userKeypairsRepository: UserKeypairsRepository, private userKeypairsRepository: UserKeypairsRepository,
) { ) {
this.cache = new RedisKVCache<MiUserKeypair>(this.redisClient, 'userKeypair', { this.cache = new MemoryKVCache<MiUserKeypair>(1000 * 60 * 60 * 24); // 24h
lifetime: 1000 * 60 * 60 * 24, // 24h
memoryCacheLifetime: 1000 * 60 * 60, // 1h
fetcher: (key) => this.userKeypairsRepository.findOneByOrFail({ userId: key }),
toRedisConverter: (value) => JSON.stringify(value),
fromRedisConverter: (value) => JSON.parse(value),
});
} }
@bindThis @bindThis
public async getUserKeypair(userId: MiUser['id']): Promise<MiUserKeypair> { public async getUserKeypair(userId: MiUser['id']): Promise<MiUserKeypair> {
return await this.cache.fetch(userId); return await this.cache.fetch(userId, () => this.userKeypairsRepository.findOneByOrFail({ userId }));
} }
@bindThis @bindThis

View file

@ -11,21 +11,22 @@ import type { MiUser } from '@/models/User.js';
import type { MiUserList } from '@/models/UserList.js'; import type { MiUserList } from '@/models/UserList.js';
import type { MiUserListMembership } from '@/models/UserListMembership.js'; import type { MiUserListMembership } from '@/models/UserListMembership.js';
import { IdService } from '@/core/IdService.js'; import { IdService } from '@/core/IdService.js';
import type { GlobalEvents } from '@/core/GlobalEventService.js'; import type { GlobalEvents, InternalEventTypes } from '@/core/GlobalEventService.js';
import { GlobalEventService } from '@/core/GlobalEventService.js'; import { GlobalEventService } from '@/core/GlobalEventService.js';
import { DI } from '@/di-symbols.js'; import { DI } from '@/di-symbols.js';
import { UserEntityService } from '@/core/entities/UserEntityService.js'; import { UserEntityService } from '@/core/entities/UserEntityService.js';
import { bindThis } from '@/decorators.js'; import { bindThis } from '@/decorators.js';
import { QueueService } from '@/core/QueueService.js'; import { QueueService } from '@/core/QueueService.js';
import { RedisKVCache } from '@/misc/cache.js'; import { QuantumKVCache } from '@/misc/QuantumKVCache.js';
import { RoleService } from '@/core/RoleService.js'; import { RoleService } from '@/core/RoleService.js';
import { SystemAccountService } from '@/core/SystemAccountService.js'; import { SystemAccountService } from '@/core/SystemAccountService.js';
import { InternalEventService } from '@/core/InternalEventService.js';
@Injectable() @Injectable()
export class UserListService implements OnApplicationShutdown, OnModuleInit { export class UserListService implements OnApplicationShutdown, OnModuleInit {
public static TooManyUsersError = class extends Error {}; public static TooManyUsersError = class extends Error {};
public membersCache: RedisKVCache<Set<string>>; public membersCache: QuantumKVCache<Set<string>>;
private roleService: RoleService; private roleService: RoleService;
constructor( constructor(
@ -48,16 +49,15 @@ export class UserListService implements OnApplicationShutdown, OnModuleInit {
private globalEventService: GlobalEventService, private globalEventService: GlobalEventService,
private queueService: QueueService, private queueService: QueueService,
private systemAccountService: SystemAccountService, private systemAccountService: SystemAccountService,
private readonly internalEventService: InternalEventService,
) { ) {
this.membersCache = new RedisKVCache<Set<string>>(this.redisClient, 'userListMembers', { this.membersCache = new QuantumKVCache<Set<string>>(this.internalEventService, 'userListMembers', {
lifetime: 1000 * 60 * 30, // 30m lifetime: 1000 * 60 * 30, // 30m
memoryCacheLifetime: 1000 * 60, // 1m
fetcher: (key) => this.userListMembershipsRepository.find({ where: { userListId: key }, select: ['userId'] }).then(xs => new Set(xs.map(x => x.userId))), fetcher: (key) => this.userListMembershipsRepository.find({ where: { userListId: key }, select: ['userId'] }).then(xs => new Set(xs.map(x => x.userId))),
toRedisConverter: (value) => JSON.stringify(Array.from(value)),
fromRedisConverter: (value) => new Set(JSON.parse(value)),
}); });
this.redisForSub.on('message', this.onMessage); this.internalEventService.on('userListMemberAdded', this.onMessage);
this.internalEventService.on('userListMemberRemoved', this.onMessage);
} }
async onModuleInit() { async onModuleInit() {
@ -65,15 +65,12 @@ export class UserListService implements OnApplicationShutdown, OnModuleInit {
} }
@bindThis @bindThis
private async onMessage(_: string, data: string): Promise<void> { private async onMessage<E extends 'userListMemberAdded' | 'userListMemberRemoved'>(body: InternalEventTypes[E], type: E): Promise<void> {
const obj = JSON.parse(data); {
if (obj.channel === 'internal') {
const { type, body } = obj.message as GlobalEvents['internal']['payload'];
switch (type) { switch (type) {
case 'userListMemberAdded': { case 'userListMemberAdded': {
const { userListId, memberId } = body; const { userListId, memberId } = body;
const members = await this.membersCache.get(userListId); const members = this.membersCache.get(userListId);
if (members) { if (members) {
members.add(memberId); members.add(memberId);
} }
@ -81,7 +78,7 @@ export class UserListService implements OnApplicationShutdown, OnModuleInit {
} }
case 'userListMemberRemoved': { case 'userListMemberRemoved': {
const { userListId, memberId } = body; const { userListId, memberId } = body;
const members = await this.membersCache.get(userListId); const members = this.membersCache.get(userListId);
if (members) { if (members) {
members.delete(memberId); members.delete(memberId);
} }
@ -150,7 +147,8 @@ export class UserListService implements OnApplicationShutdown, OnModuleInit {
@bindThis @bindThis
public dispose(): void { public dispose(): void {
this.redisForSub.off('message', this.onMessage); this.internalEventService.off('userListMemberAdded', this.onMessage);
this.internalEventService.off('userListMemberRemoved', this.onMessage);
this.membersCache.dispose(); this.membersCache.dispose();
} }

View file

@ -32,7 +32,7 @@ export class UserMutingService {
muteeId: target.id, muteeId: target.id,
}); });
this.cacheService.userMutingsCache.refresh(user.id); await this.cacheService.userMutingsCache.delete(user.id);
} }
@bindThis @bindThis
@ -43,9 +43,6 @@ export class UserMutingService {
id: In(mutings.map(m => m.id)), id: In(mutings.map(m => m.id)),
}); });
const muterIds = [...new Set(mutings.map(m => m.muterId))]; await this.cacheService.userMutingsCache.deleteMany(mutings.map(m => m.muterId));
for (const muterId of muterIds) {
this.cacheService.userMutingsCache.refresh(muterId);
}
} }
} }

View file

@ -33,7 +33,7 @@ export class UserRenoteMutingService {
muteeId: target.id, muteeId: target.id,
}); });
await this.cacheService.renoteMutingsCache.refresh(user.id); await this.cacheService.renoteMutingsCache.delete(user.id);
} }
@bindThis @bindThis
@ -44,9 +44,6 @@ export class UserRenoteMutingService {
id: In(mutings.map(m => m.id)), id: In(mutings.map(m => m.id)),
}); });
const muterIds = [...new Set(mutings.map(m => m.muterId))]; await this.cacheService.renoteMutingsCache.deleteMany(mutings.map(m => m.muterId));
for (const muterId of muterIds) {
await this.cacheService.renoteMutingsCache.refresh(muterId);
}
} }
} }

View file

@ -10,6 +10,7 @@ import { DI } from '@/di-symbols.js';
import { bindThis } from '@/decorators.js'; import { bindThis } from '@/decorators.js';
import { SystemWebhookService } from '@/core/SystemWebhookService.js'; import { SystemWebhookService } from '@/core/SystemWebhookService.js';
import { UserEntityService } from '@/core/entities/UserEntityService.js'; import { UserEntityService } from '@/core/entities/UserEntityService.js';
import { CacheService } from '@/core/CacheService.js';
@Injectable() @Injectable()
export class UserService { export class UserService {
@ -20,6 +21,7 @@ export class UserService {
private followingsRepository: FollowingsRepository, private followingsRepository: FollowingsRepository,
private systemWebhookService: SystemWebhookService, private systemWebhookService: SystemWebhookService,
private userEntityService: UserEntityService, private userEntityService: UserEntityService,
private readonly cacheService: CacheService,
) { ) {
} }
@ -38,14 +40,17 @@ export class UserService {
}); });
const wokeUp = result.isHibernated; const wokeUp = result.isHibernated;
if (wokeUp) { if (wokeUp) {
await Promise.all([
this.usersRepository.update(user.id, { this.usersRepository.update(user.id, {
isHibernated: false, isHibernated: false,
}); }),
this.followingsRepository.update({ this.followingsRepository.update({
followerId: user.id, followerId: user.id,
}, { }, {
isFollowerHibernated: false, isFollowerHibernated: false,
}); }),
this.cacheService.hibernatedUserCache.set(user.id, false),
]);
} }
} else { } else {
this.usersRepository.update(user.id, { this.usersRepository.update(user.id, {

View file

@ -16,6 +16,7 @@ import { bindThis } from '@/decorators.js';
import { RelationshipJobData } from '@/queue/types.js'; import { RelationshipJobData } from '@/queue/types.js';
import { ModerationLogService } from '@/core/ModerationLogService.js'; import { ModerationLogService } from '@/core/ModerationLogService.js';
import { isSystemAccount } from '@/misc/is-system-account.js'; import { isSystemAccount } from '@/misc/is-system-account.js';
import { CacheService } from '@/core/CacheService.js';
@Injectable() @Injectable()
export class UserSuspendService { export class UserSuspendService {
@ -34,6 +35,7 @@ export class UserSuspendService {
private globalEventService: GlobalEventService, private globalEventService: GlobalEventService,
private apRendererService: ApRendererService, private apRendererService: ApRendererService,
private moderationLogService: ModerationLogService, private moderationLogService: ModerationLogService,
private readonly cacheService: CacheService,
) { ) {
} }
@ -143,12 +145,8 @@ export class UserSuspendService {
@bindThis @bindThis
private async unFollowAll(follower: MiUser) { private async unFollowAll(follower: MiUser) {
const followings = await this.followingsRepository.find({ const followings = await this.cacheService.userFollowingsCache.fetch(follower.id)
where: { .then(fs => Array.from(fs.values()).filter(f => f.followeeHost != null));
followerId: follower.id,
followeeId: Not(IsNull()),
},
});
const jobs: RelationshipJobData[] = []; const jobs: RelationshipJobData[] = [];
for (const following of followings) { for (const following of followings) {

View file

@ -5,7 +5,6 @@
import { Inject, Injectable } from '@nestjs/common'; import { Inject, Injectable } from '@nestjs/common';
import { IsNull, Not } from 'typeorm'; import { IsNull, Not } from 'typeorm';
import { UnrecoverableError } from 'bullmq';
import { DI } from '@/di-symbols.js'; import { DI } from '@/di-symbols.js';
import type { FollowingsRepository } from '@/models/_.js'; import type { FollowingsRepository } from '@/models/_.js';
import type { MiLocalUser, MiRemoteUser, MiUser } from '@/models/User.js'; import type { MiLocalUser, MiRemoteUser, MiUser } from '@/models/User.js';
@ -14,6 +13,7 @@ import { UserEntityService } from '@/core/entities/UserEntityService.js';
import { bindThis } from '@/decorators.js'; import { bindThis } from '@/decorators.js';
import type { IActivity } from '@/core/activitypub/type.js'; import type { IActivity } from '@/core/activitypub/type.js';
import { ThinUser } from '@/queue/types.js'; import { ThinUser } from '@/queue/types.js';
import { CacheService } from '@/core/CacheService.js';
interface IRecipe { interface IRecipe {
type: string; type: string;
@ -41,16 +41,14 @@ class DeliverManager {
/** /**
* Constructor * Constructor
* @param userEntityService
* @param followingsRepository
* @param queueService * @param queueService
* @param cacheService
* @param actor Actor * @param actor Actor
* @param activity Activity to deliver * @param activity Activity to deliver
*/ */
constructor( constructor(
private userEntityService: UserEntityService,
private followingsRepository: FollowingsRepository,
private queueService: QueueService, private queueService: QueueService,
private readonly cacheService: CacheService,
actor: { id: MiUser['id']; host: null; }, actor: { id: MiUser['id']; host: null; },
activity: IActivity | null, activity: IActivity | null,
@ -114,24 +112,23 @@ class DeliverManager {
// Process follower recipes first to avoid duplication when processing direct recipes later. // Process follower recipes first to avoid duplication when processing direct recipes later.
if (this.recipes.some(r => isFollowers(r))) { if (this.recipes.some(r => isFollowers(r))) {
// followers deliver // followers deliver
// TODO: SELECT DISTINCT ON ("followerSharedInbox") "followerSharedInbox" みたいな問い合わせにすればよりパフォーマンス向上できそう
// ただ、sharedInboxがnullなリモートユーザーも稀におり、その対応ができなさそう // ただ、sharedInboxがnullなリモートユーザーも稀におり、その対応ができなさそう
const followers = await this.followingsRepository.find({ const followers = await this.cacheService.userFollowersCache
where: { .fetch(this.actor.id)
followeeId: this.actor.id, .then(f => Array
followerHost: Not(IsNull()), .from(f.values())
}, .filter(f => f.followerHost != null)
select: { .map(f => ({
followerSharedInbox: true, followerInbox: f.followerInbox,
followerInbox: true, followerSharedInbox: f.followerSharedInbox,
followerId: true, })));
},
});
for (const following of followers) { for (const following of followers) {
const inbox = following.followerSharedInbox ?? following.followerInbox; if (following.followerSharedInbox) {
if (inbox === null) throw new UnrecoverableError(`deliver failed for ${this.actor.id}: follower ${following.followerId} inbox is null`); inboxes.set(following.followerSharedInbox, true);
inboxes.set(inbox, following.followerSharedInbox != null); } else if (following.followerInbox) {
inboxes.set(following.followerInbox, false);
}
} }
} }
@ -153,11 +150,8 @@ class DeliverManager {
@Injectable() @Injectable()
export class ApDeliverManagerService { export class ApDeliverManagerService {
constructor( constructor(
@Inject(DI.followingsRepository)
private followingsRepository: FollowingsRepository,
private userEntityService: UserEntityService,
private queueService: QueueService, private queueService: QueueService,
private readonly cacheService: CacheService,
) { ) {
} }
@ -169,9 +163,8 @@ export class ApDeliverManagerService {
@bindThis @bindThis
public async deliverToFollowers(actor: { id: MiLocalUser['id']; host: null; }, activity: IActivity): Promise<void> { public async deliverToFollowers(actor: { id: MiLocalUser['id']; host: null; }, activity: IActivity): Promise<void> {
const manager = new DeliverManager( const manager = new DeliverManager(
this.userEntityService,
this.followingsRepository,
this.queueService, this.queueService,
this.cacheService,
actor, actor,
activity, activity,
); );
@ -188,9 +181,8 @@ export class ApDeliverManagerService {
@bindThis @bindThis
public async deliverToUser(actor: { id: MiLocalUser['id']; host: null; }, activity: IActivity, to: MiRemoteUser): Promise<void> { public async deliverToUser(actor: { id: MiLocalUser['id']; host: null; }, activity: IActivity, to: MiRemoteUser): Promise<void> {
const manager = new DeliverManager( const manager = new DeliverManager(
this.userEntityService,
this.followingsRepository,
this.queueService, this.queueService,
this.cacheService,
actor, actor,
activity, activity,
); );
@ -207,9 +199,8 @@ export class ApDeliverManagerService {
@bindThis @bindThis
public async deliverToUsers(actor: { id: MiLocalUser['id']; host: null; }, activity: IActivity, targets: MiRemoteUser[]): Promise<void> { public async deliverToUsers(actor: { id: MiLocalUser['id']; host: null; }, activity: IActivity, targets: MiRemoteUser[]): Promise<void> {
const manager = new DeliverManager( const manager = new DeliverManager(
this.userEntityService,
this.followingsRepository,
this.queueService, this.queueService,
this.cacheService,
actor, actor,
activity, activity,
); );
@ -220,9 +211,8 @@ export class ApDeliverManagerService {
@bindThis @bindThis
public createDeliverManager(actor: { id: MiUser['id']; host: null; }, activity: IActivity | null): DeliverManager { public createDeliverManager(actor: { id: MiUser['id']; host: null; }, activity: IActivity | null): DeliverManager {
return new DeliverManager( return new DeliverManager(
this.userEntityService,
this.followingsRepository,
this.queueService, this.queueService,
this.cacheService,
actor, actor,
activity, activity,

View file

@ -37,6 +37,7 @@ import InstanceChart from '@/core/chart/charts/instance.js';
import FederationChart from '@/core/chart/charts/federation.js'; import FederationChart from '@/core/chart/charts/federation.js';
import { FetchInstanceMetadataService } from '@/core/FetchInstanceMetadataService.js'; import { FetchInstanceMetadataService } from '@/core/FetchInstanceMetadataService.js';
import { UpdateInstanceQueue } from '@/core/UpdateInstanceQueue.js'; import { UpdateInstanceQueue } from '@/core/UpdateInstanceQueue.js';
import { CacheService } from '@/core/CacheService.js';
import { getApHrefNullable, getApId, getApIds, getApType, getNullableApId, isAccept, isActor, isAdd, isAnnounce, isApObject, isBlock, isCollectionOrOrderedCollection, isCreate, isDelete, isFlag, isFollow, isLike, isDislike, isMove, isPost, isReject, isRemove, isTombstone, isUndo, isUpdate, validActor, validPost, isActivity, IObjectWithId } from './type.js'; import { getApHrefNullable, getApId, getApIds, getApType, getNullableApId, isAccept, isActor, isAdd, isAnnounce, isApObject, isBlock, isCollectionOrOrderedCollection, isCreate, isDelete, isFlag, isFollow, isLike, isDislike, isMove, isPost, isReject, isRemove, isTombstone, isUndo, isUpdate, validActor, validPost, isActivity, IObjectWithId } from './type.js';
import { ApNoteService } from './models/ApNoteService.js'; import { ApNoteService } from './models/ApNoteService.js';
import { ApLoggerService } from './ApLoggerService.js'; import { ApLoggerService } from './ApLoggerService.js';
@ -98,6 +99,7 @@ export class ApInboxService {
private readonly instanceChart: InstanceChart, private readonly instanceChart: InstanceChart,
private readonly federationChart: FederationChart, private readonly federationChart: FederationChart,
private readonly updateInstanceQueue: UpdateInstanceQueue, private readonly updateInstanceQueue: UpdateInstanceQueue,
private readonly cacheService: CacheService,
) { ) {
this.logger = this.apLoggerService.logger; this.logger = this.apLoggerService.logger;
} }
@ -365,7 +367,7 @@ export class ApInboxService {
const renote = await this.apNoteService.resolveNote(target, { resolver, sentFrom: getApId(target) }); const renote = await this.apNoteService.resolveNote(target, { resolver, sentFrom: getApId(target) });
if (renote == null) return 'announce target is null'; if (renote == null) return 'announce target is null';
if (!await this.noteEntityService.isVisibleForMe(renote, actor.id)) { if (!await this.noteEntityService.isVisibleForMe(renote, actor.id, { me: actor })) {
return 'skip: invalid actor for this activity'; return 'skip: invalid actor for this activity';
} }
@ -766,12 +768,7 @@ export class ApInboxService {
return 'skip: follower not found'; return 'skip: follower not found';
} }
const isFollowing = await this.followingsRepository.exists({ const isFollowing = await this.cacheService.userFollowingsCache.fetch(follower.id).then(f => f.has(actor.id));
where: {
followerId: follower.id,
followeeId: actor.id,
},
});
if (isFollowing) { if (isFollowing) {
await this.userFollowingService.unfollow(follower, actor); await this.userFollowingService.unfollow(follower, actor);
@ -830,12 +827,7 @@ export class ApInboxService {
}, },
}); });
const isFollowing = await this.followingsRepository.exists({ const isFollowing = await this.cacheService.userFollowingsCache.fetch(actor.id).then(f => f.has(followee.id));
where: {
followerId: actor.id,
followeeId: followee.id,
},
});
if (requestExist) { if (requestExist) {
await this.userFollowingService.cancelFollowRequest(followee, actor); await this.userFollowingService.cancelFollowRequest(followee, actor);

View file

@ -741,11 +741,18 @@ export class ApPersonService implements OnModuleInit, OnApplicationShutdown {
this.hashtagService.updateUsertags(exist, tags); this.hashtagService.updateUsertags(exist, tags);
// 該当ユーザーが既にフォロワーになっていた場合はFollowingもアップデートする // 該当ユーザーが既にフォロワーになっていた場合はFollowingもアップデートする
if (exist.inbox !== person.inbox || exist.sharedInbox !== (person.sharedInbox ?? person.endpoints?.sharedInbox)) {
await this.followingsRepository.update( await this.followingsRepository.update(
{ followerId: exist.id }, { followerId: exist.id },
{ followerSharedInbox: person.sharedInbox ?? person.endpoints?.sharedInbox ?? null }, {
followerInbox: person.inbox,
followerSharedInbox: person.sharedInbox ?? person.endpoints?.sharedInbox ?? null,
},
); );
await this.cacheService.refreshFollowRelationsFor(exist.id);
}
await this.updateFeatured(exist.id, resolver).catch(err => { await this.updateFeatured(exist.id, resolver).catch(err => {
// Permanent error implies hidden or inaccessible, which is a normal thing. // Permanent error implies hidden or inaccessible, which is a normal thing.
if (isRetryableError(err)) { if (isRetryableError(err)) {

View file

@ -44,6 +44,7 @@ export default class FederationChart extends Chart<typeof schema> { // eslint-di
} }
protected async tickMinor(): Promise<Partial<KVs<typeof schema>>> { protected async tickMinor(): Promise<Partial<KVs<typeof schema>>> {
// TODO optimization: replace these with exists()
const pubsubSubQuery = this.followingsRepository.createQueryBuilder('f') const pubsubSubQuery = this.followingsRepository.createQueryBuilder('f')
.select('f.followerHost') .select('f.followerHost')
.where('f.followerHost IS NOT NULL'); .where('f.followerHost IS NOT NULL');

View file

@ -15,6 +15,7 @@ import Chart from '../core.js';
import { ChartLoggerService } from '../ChartLoggerService.js'; import { ChartLoggerService } from '../ChartLoggerService.js';
import { name, schema } from './entities/per-user-following.js'; import { name, schema } from './entities/per-user-following.js';
import type { KVs } from '../core.js'; import type { KVs } from '../core.js';
import { CacheService } from '@/core/CacheService.js';
/** /**
* *
@ -31,23 +32,25 @@ export default class PerUserFollowingChart extends Chart<typeof schema> { // esl
private appLockService: AppLockService, private appLockService: AppLockService,
private userEntityService: UserEntityService, private userEntityService: UserEntityService,
private chartLoggerService: ChartLoggerService, private chartLoggerService: ChartLoggerService,
private readonly cacheService: CacheService,
) { ) {
super(db, (k) => appLockService.getChartInsertLock(k), chartLoggerService.logger, name, schema, true); super(db, (k) => appLockService.getChartInsertLock(k), chartLoggerService.logger, name, schema, true);
} }
protected async tickMajor(group: string): Promise<Partial<KVs<typeof schema>>> { protected async tickMajor(group: string): Promise<Partial<KVs<typeof schema>>> {
const [ const [
localFollowingsCount, followees,
localFollowersCount, followers,
remoteFollowingsCount,
remoteFollowersCount,
] = await Promise.all([ ] = await Promise.all([
this.followingsRepository.countBy({ followerId: group, followeeHost: IsNull() }), this.cacheService.userFollowingsCache.fetch(group).then(fs => Array.from(fs.values())),
this.followingsRepository.countBy({ followeeId: group, followerHost: IsNull() }), this.cacheService.userFollowersCache.fetch(group).then(fs => Array.from(fs.values())),
this.followingsRepository.countBy({ followerId: group, followeeHost: Not(IsNull()) }),
this.followingsRepository.countBy({ followeeId: group, followerHost: Not(IsNull()) }),
]); ]);
const localFollowingsCount = followees.reduce((sum, f) => sum + (f.followeeHost == null ? 1 : 0), 0);
const localFollowersCount = followers.reduce((sum, f) => sum + (f.followerHost == null ? 1 : 0), 0);
const remoteFollowingsCount = followees.reduce((sum, f) => sum + (f.followeeHost == null ? 0 : 1), 0);
const remoteFollowersCount = followers.reduce((sum, f) => sum + (f.followerHost == null ? 0 : 1), 0);
return { return {
'local.followings.total': localFollowingsCount, 'local.followings.total': localFollowingsCount,
'local.followers.total': localFollowersCount, 'local.followers.total': localFollowersCount,

View file

@ -11,7 +11,7 @@ import type { Packed } from '@/misc/json-schema.js';
import { awaitAll } from '@/misc/prelude/await-all.js'; import { awaitAll } from '@/misc/prelude/await-all.js';
import type { MiUser } from '@/models/User.js'; import type { MiUser } from '@/models/User.js';
import type { MiNote } from '@/models/Note.js'; import type { MiNote } from '@/models/Note.js';
import type { UsersRepository, NotesRepository, FollowingsRepository, PollsRepository, PollVotesRepository, NoteReactionsRepository, ChannelsRepository, MiMeta } from '@/models/_.js'; import type { UsersRepository, NotesRepository, FollowingsRepository, PollsRepository, PollVotesRepository, NoteReactionsRepository, ChannelsRepository, MiMeta, MiPollVote, MiPoll, MiChannel, MiFollowing } from '@/models/_.js';
import { bindThis } from '@/decorators.js'; import { bindThis } from '@/decorators.js';
import { DebounceLoader } from '@/misc/loader.js'; import { DebounceLoader } from '@/misc/loader.js';
import { IdService } from '@/core/IdService.js'; import { IdService } from '@/core/IdService.js';
@ -26,13 +26,13 @@ import type { UserEntityService } from './UserEntityService.js';
import type { DriveFileEntityService } from './DriveFileEntityService.js'; import type { DriveFileEntityService } from './DriveFileEntityService.js';
// is-renote.tsとよしなにリンク // is-renote.tsとよしなにリンク
function isPureRenote(note: MiNote): note is MiNote & { renoteId: MiNote['id']; renote: MiNote } { function isPureRenote(note: MiNote): note is MiNote & { renoteId: MiNote['id'] } {
return ( return (
note.renote != null && note.renoteId != null &&
note.reply == null && note.replyId == null &&
note.text == null && note.text == null &&
note.cw == null && note.cw == null &&
(note.fileIds == null || note.fileIds.length === 0) && note.fileIds.length === 0 &&
!note.hasPoll !note.hasPoll
); );
} }
@ -132,7 +132,10 @@ export class NoteEntityService implements OnModuleInit {
} }
@bindThis @bindThis
public async hideNote(packedNote: Packed<'Note'>, meId: MiUser['id'] | null): Promise<void> { public async hideNote(packedNote: Packed<'Note'>, meId: MiUser['id'] | null, hint?: {
myFollowing?: ReadonlyMap<string, unknown>,
myBlockers?: ReadonlySet<string>,
}): Promise<void> {
if (meId === packedNote.userId) return; if (meId === packedNote.userId) return;
// TODO: isVisibleForMe を使うようにしても良さそう(型違うけど) // TODO: isVisibleForMe を使うようにしても良さそう(型違うけど)
@ -188,14 +191,9 @@ export class NoteEntityService implements OnModuleInit {
} else if (packedNote.renote && (meId === packedNote.renote.userId)) { } else if (packedNote.renote && (meId === packedNote.renote.userId)) {
hide = false; hide = false;
} else { } else {
// フォロワーかどうか const isFollowing = hint?.myFollowing
// TODO: 当関数呼び出しごとにクエリが走るのは重そうだからなんとかする ? hint.myFollowing.has(packedNote.userId)
const isFollowing = await this.followingsRepository.exists({ : (await this.cacheService.userFollowingsCache.fetch(meId)).has(packedNote.userId);
where: {
followeeId: packedNote.userId,
followerId: meId,
},
});
hide = !isFollowing; hide = !isFollowing;
} }
@ -211,7 +209,8 @@ export class NoteEntityService implements OnModuleInit {
} }
if (!hide && meId && packedNote.userId !== meId) { if (!hide && meId && packedNote.userId !== meId) {
const isBlocked = (await this.cacheService.userBlockedCache.fetch(meId)).has(packedNote.userId); const blockers = hint?.myBlockers ?? await this.cacheService.userBlockedCache.fetch(meId);
const isBlocked = blockers.has(packedNote.userId);
if (isBlocked) hide = true; if (isBlocked) hide = true;
} }
@ -235,8 +234,11 @@ export class NoteEntityService implements OnModuleInit {
} }
@bindThis @bindThis
private async populatePoll(note: MiNote, meId: MiUser['id'] | null) { private async populatePoll(note: MiNote, meId: MiUser['id'] | null, hint?: {
const poll = await this.pollsRepository.findOneByOrFail({ noteId: note.id }); poll?: MiPoll,
myVotes?: MiPollVote[],
}) {
const poll = hint?.poll ?? await this.pollsRepository.findOneByOrFail({ noteId: note.id });
const choices = poll.choices.map(c => ({ const choices = poll.choices.map(c => ({
text: c, text: c,
votes: poll.votes[poll.choices.indexOf(c)], votes: poll.votes[poll.choices.indexOf(c)],
@ -245,7 +247,7 @@ export class NoteEntityService implements OnModuleInit {
if (meId) { if (meId) {
if (poll.multiple) { if (poll.multiple) {
const votes = await this.pollVotesRepository.findBy({ const votes = hint?.myVotes ?? await this.pollVotesRepository.findBy({
userId: meId, userId: meId,
noteId: note.id, noteId: note.id,
}); });
@ -255,7 +257,7 @@ export class NoteEntityService implements OnModuleInit {
choices[myChoice].isVoted = true; choices[myChoice].isVoted = true;
} }
} else { } else {
const vote = await this.pollVotesRepository.findOneBy({ const vote = hint?.myVotes ? hint.myVotes[0] : await this.pollVotesRepository.findOneBy({
userId: meId, userId: meId,
noteId: note.id, noteId: note.id,
}); });
@ -317,7 +319,12 @@ export class NoteEntityService implements OnModuleInit {
} }
@bindThis @bindThis
public async isVisibleForMe(note: MiNote, meId: MiUser['id'] | null): Promise<boolean> { public async isVisibleForMe(note: MiNote, meId: MiUser['id'] | null, hint?: {
myFollowing?: ReadonlySet<string>,
myBlocking?: ReadonlySet<string>,
myBlockers?: ReadonlySet<string>,
me?: Pick<MiUser, 'host'> | null,
}): Promise<boolean> {
// This code must always be synchronized with the checks in generateVisibilityQuery. // This code must always be synchronized with the checks in generateVisibilityQuery.
// visibility が specified かつ自分が指定されていなかったら非表示 // visibility が specified かつ自分が指定されていなかったら非表示
if (note.visibility === 'specified') { if (note.visibility === 'specified') {
@ -345,16 +352,16 @@ export class NoteEntityService implements OnModuleInit {
return true; return true;
} else { } else {
// フォロワーかどうか // フォロワーかどうか
const [blocked, following, user] = await Promise.all([ const [blocked, following, userHost] = await Promise.all([
this.cacheService.userBlockingCache.fetch(meId).then((ids) => ids.has(note.userId)), hint?.myBlocking
this.followingsRepository.count({ ? hint.myBlocking.has(note.userId)
where: { : this.cacheService.userBlockingCache.fetch(meId).then((ids) => ids.has(note.userId)),
followeeId: note.userId, hint?.myFollowing
followerId: meId, ? hint.myFollowing.has(note.userId)
}, : this.cacheService.userFollowingsCache.fetch(meId).then(ids => ids.has(note.userId)),
take: 1, hint?.me !== undefined
}), ? (hint.me?.host ?? null)
this.usersRepository.findOneByOrFail({ id: meId }), : this.cacheService.findUserById(meId).then(me => me.host),
]); ]);
if (blocked) return false; if (blocked) return false;
@ -366,12 +373,13 @@ export class NoteEntityService implements OnModuleInit {
in which case we can never know the following. Instead we have in which case we can never know the following. Instead we have
to assume that the users are following each other. to assume that the users are following each other.
*/ */
return following > 0 || (note.userHost != null && user.host != null); return following || (note.userHost != null && userHost != null);
} }
} }
if (meId != null) { if (meId != null) {
const isBlocked = (await this.cacheService.userBlockedCache.fetch(meId)).has(note.userId); const blockers = hint?.myBlockers ?? await this.cacheService.userBlockedCache.fetch(meId);
const isBlocked = blockers.has(note.userId);
if (isBlocked) return false; if (isBlocked) return false;
} }
@ -408,6 +416,12 @@ export class NoteEntityService implements OnModuleInit {
packedFiles: Map<MiNote['fileIds'][number], Packed<'DriveFile'> | null>; packedFiles: Map<MiNote['fileIds'][number], Packed<'DriveFile'> | null>;
packedUsers: Map<MiUser['id'], Packed<'UserLite'>>; packedUsers: Map<MiUser['id'], Packed<'UserLite'>>;
mentionHandles: Record<string, string | undefined>; mentionHandles: Record<string, string | undefined>;
userFollowings: Map<string, Map<string, Omit<MiFollowing, 'isFollowerHibernated'>>>;
userBlockers: Map<string, Set<string>>;
polls: Map<string, MiPoll>;
pollVotes: Map<string, Map<string, MiPollVote[]>>;
channels: Map<string, MiChannel>;
notes: Map<string, MiNote>;
}; };
}, },
): Promise<Packed<'Note'>> { ): Promise<Packed<'Note'>> {
@ -437,9 +451,7 @@ export class NoteEntityService implements OnModuleInit {
} }
const channel = note.channelId const channel = note.channelId
? note.channel ? (opts._hint_?.channels.get(note.channelId) ?? note.channel ?? await this.channelsRepository.findOneBy({ id: note.channelId }))
? note.channel
: await this.channelsRepository.findOneBy({ id: note.channelId })
: null; : null;
const reactionEmojiNames = Object.keys(reactions) const reactionEmojiNames = Object.keys(reactions)
@ -485,7 +497,10 @@ export class NoteEntityService implements OnModuleInit {
mentionHandles: note.mentions.length > 0 ? this.getUserHandles(note.mentions, options?._hint_?.mentionHandles) : undefined, mentionHandles: note.mentions.length > 0 ? this.getUserHandles(note.mentions, options?._hint_?.mentionHandles) : undefined,
uri: note.uri ?? undefined, uri: note.uri ?? undefined,
url: note.url ?? undefined, url: note.url ?? undefined,
poll: note.hasPoll ? this.populatePoll(note, meId) : undefined, poll: note.hasPoll ? this.populatePoll(note, meId, {
poll: opts._hint_?.polls.get(note.id),
myVotes: opts._hint_?.pollVotes.get(note.id)?.get(note.userId),
}) : undefined,
...(meId && Object.keys(reactions).length > 0 ? { ...(meId && Object.keys(reactions).length > 0 ? {
myReaction: this.populateMyReaction({ myReaction: this.populateMyReaction({
@ -499,14 +514,14 @@ export class NoteEntityService implements OnModuleInit {
clippedCount: note.clippedCount, clippedCount: note.clippedCount,
processErrors: note.processErrors, processErrors: note.processErrors,
reply: note.replyId ? this.pack(note.reply ?? note.replyId, me, { reply: note.replyId ? this.pack(note.reply ?? opts._hint_?.notes.get(note.replyId) ?? note.replyId, me, {
detail: false, detail: false,
skipHide: opts.skipHide, skipHide: opts.skipHide,
withReactionAndUserPairCache: opts.withReactionAndUserPairCache, withReactionAndUserPairCache: opts.withReactionAndUserPairCache,
_hint_: options?._hint_, _hint_: options?._hint_,
}) : undefined, }) : undefined,
renote: note.renoteId ? this.pack(note.renote ?? note.renoteId, me, { renote: note.renoteId ? this.pack(note.renote ?? opts._hint_?.notes.get(note.renoteId) ?? note.renoteId, me, {
detail: true, detail: true,
skipHide: opts.skipHide, skipHide: opts.skipHide,
withReactionAndUserPairCache: opts.withReactionAndUserPairCache, withReactionAndUserPairCache: opts.withReactionAndUserPairCache,
@ -518,7 +533,10 @@ export class NoteEntityService implements OnModuleInit {
this.treatVisibility(packed); this.treatVisibility(packed);
if (!opts.skipHide) { if (!opts.skipHide) {
await this.hideNote(packed, meId); await this.hideNote(packed, meId, meId == null ? undefined : {
myFollowing: opts._hint_?.userFollowings.get(meId),
myBlockers: opts._hint_?.userBlockers.get(meId),
});
} }
return packed; return packed;
@ -535,79 +553,139 @@ export class NoteEntityService implements OnModuleInit {
) { ) {
if (notes.length === 0) return []; if (notes.length === 0) return [];
const targetNotes: MiNote[] = []; const targetNotesMap = new Map<string, MiNote>();
const targetNotesToFetch : string[] = [];
for (const note of notes) { for (const note of notes) {
if (isPureRenote(note)) { if (isPureRenote(note)) {
// we may need to fetch 'my reaction' for renote target. // we may need to fetch 'my reaction' for renote target.
targetNotes.push(note.renote); if (note.renote) {
targetNotesMap.set(note.renote.id, note.renote);
if (note.renote.reply) { if (note.renote.reply) {
// idem if the renote is also a reply. // idem if the renote is also a reply.
targetNotes.push(note.renote.reply); targetNotesMap.set(note.renote.reply.id, note.renote.reply);
}
} else if (options?.detail) {
targetNotesToFetch.push(note.renoteId);
} }
} else { } else {
if (note.reply) { if (note.reply) {
// idem for OP of a regular reply. // idem for OP of a regular reply.
targetNotes.push(note.reply); targetNotesMap.set(note.reply.id, note.reply);
} else if (note.replyId && options?.detail) {
targetNotesToFetch.push(note.replyId);
} }
targetNotes.push(note); targetNotesMap.set(note.id, note);
} }
} }
const bufferedReactions = this.meta.enableReactionsBuffering ? await this.reactionsBufferingService.getMany([...getAppearNoteIds(notes)]) : null; // Don't fetch notes that were added by ID and then found inline in another note.
for (let i = targetNotesToFetch.length - 1; i >= 0; i--) {
const meId = me ? me.id : null; if (targetNotesMap.has(targetNotesToFetch[i])) {
const myReactionsMap = new Map<MiNote['id'], string | null>(); targetNotesToFetch.splice(i, 1);
if (meId) {
const idsNeedFetchMyReaction = new Set<MiNote['id']>();
for (const note of targetNotes) {
const reactionsCount = Object.values(this.reactionsBufferingService.mergeReactions(note.reactions, bufferedReactions?.get(note.id)?.deltas ?? {})).reduce((a, b) => a + b, 0);
if (reactionsCount === 0) {
myReactionsMap.set(note.id, null);
} else if (reactionsCount <= note.reactionAndUserPairCache.length + (bufferedReactions?.get(note.id)?.pairs.length ?? 0)) {
const pairInBuffer = bufferedReactions?.get(note.id)?.pairs.find(p => p[0] === meId);
if (pairInBuffer) {
myReactionsMap.set(note.id, pairInBuffer[1]);
} else {
const pair = note.reactionAndUserPairCache.find(p => p.startsWith(meId));
myReactionsMap.set(note.id, pair ? pair.split('/')[1] : null);
}
} else {
idsNeedFetchMyReaction.add(note.id);
} }
} }
const myReactions = idsNeedFetchMyReaction.size > 0 ? await this.noteReactionsRepository.findBy({ // Populate any relations that weren't included in the source
userId: meId, if (targetNotesToFetch.length > 0) {
noteId: In(Array.from(idsNeedFetchMyReaction)), const newNotes = await this.notesRepository.find({
}) : []; where: {
id: In(targetNotesToFetch),
},
relations: {
user: {
userProfile: true,
},
reply: {
user: {
userProfile: true,
},
},
renote: {
user: {
userProfile: true,
},
reply: {
user: {
userProfile: true,
},
},
},
channel: true,
},
});
for (const id of idsNeedFetchMyReaction) { for (const note of newNotes) {
myReactionsMap.set(id, myReactions.find(reaction => reaction.noteId === id)?.reaction ?? null); targetNotesMap.set(note.id, note);
} }
} }
await this.customEmojiService.prefetchEmojis(this.aggregateNoteEmojis(notes)); const targetNotes = Array.from(targetNotesMap.values());
// TODO: 本当は renote とか reply がないのに renoteId とか replyId があったらここで解決しておく const noteIds = Array.from(targetNotesMap.keys());
const fileIds = notes.map(n => [n.fileIds, n.renote?.fileIds, n.reply?.fileIds]).flat(2).filter(x => x != null);
const packedFiles = fileIds.length > 0 ? await this.driveFileEntityService.packManyByIdsMap(fileIds) : new Map();
const users = [
...notes.map(({ user, userId }) => user ?? userId),
...notes.map(({ replyUserId }) => replyUserId).filter(x => x != null),
...notes.map(({ renoteUserId }) => renoteUserId).filter(x => x != null),
];
const packedUsers = await this.userEntityService.packMany(users, me)
.then(users => new Map(users.map(u => [u.id, u])));
// Recursively add all mentioned users from all notes + replies + renotes const usersMap = new Map<string, MiUser | string>();
const allMentionedUsers = targetNotes.reduce((users, note) => { const allUsers = notes.flatMap(note => [
for (const user of note.mentions) { note.user ?? note.userId,
users.add(user); note.reply?.user ?? note.replyUserId,
note.renote?.user ?? note.renoteUserId,
]);
for (const user of allUsers) {
if (!user) continue;
if (typeof(user) === 'object') {
// ID -> Entity
usersMap.set(user.id, user);
} else if (!usersMap.has(user)) {
// ID -> ID
usersMap.set(user, user);
} }
return users; }
}, new Set<string>());
const mentionHandles = await this.getUserHandles(Array.from(allMentionedUsers)); const users = Array.from(usersMap.values());
const userIds = Array.from(usersMap.keys());
const fileIds = new Set(targetNotes.flatMap(n => n.fileIds));
const mentionedUsers = new Set(targetNotes.flatMap(note => note.mentions));
const [{ bufferedReactions, myReactionsMap }, packedFiles, packedUsers, mentionHandles, userFollowings, userBlockers, polls, pollVotes, channels] = await Promise.all([
// bufferedReactions & myReactionsMap
this.getReactions(targetNotes, me),
// packedFiles
this.driveFileEntityService.packManyByIdsMap(Array.from(fileIds)),
// packedUsers
this.userEntityService.packMany(users, me)
.then(users => new Map(users.map(u => [u.id, u]))),
// mentionHandles
this.getUserHandles(Array.from(mentionedUsers)),
// userFollowings
this.cacheService.userFollowingsCache.fetchMany(userIds).then(fs => new Map(fs)),
// userBlockers
this.cacheService.userBlockedCache.fetchMany(userIds).then(bs => new Map(bs)),
// polls
this.pollsRepository.findBy({ noteId: In(noteIds) })
.then(polls => new Map(polls.map(p => [p.noteId, p]))),
// pollVotes
this.pollVotesRepository.findBy({ noteId: In(noteIds), userId: In(userIds) })
.then(votes => votes.reduce((noteMap, vote) => {
let userMap = noteMap.get(vote.noteId);
if (!userMap) {
userMap = new Map<string, MiPollVote[]>();
noteMap.set(vote.noteId, userMap);
}
let voteList = userMap.get(vote.userId);
if (!voteList) {
voteList = [];
userMap.set(vote.userId, voteList);
}
voteList.push(vote);
return noteMap;
}, new Map<string, Map<string, MiPollVote[]>>)),
// channels
this.getChannels(targetNotes),
// (not returned)
this.customEmojiService.prefetchEmojis(this.aggregateNoteEmojis(notes)),
]);
return await Promise.all(notes.map(n => this.pack(n, me, { return await Promise.all(notes.map(n => this.pack(n, me, {
...options, ...options,
@ -617,6 +695,12 @@ export class NoteEntityService implements OnModuleInit {
packedFiles, packedFiles,
packedUsers, packedUsers,
mentionHandles, mentionHandles,
userFollowings,
userBlockers,
polls,
pollVotes,
channels,
notes: new Map(targetNotes.map(n => [n.id, n])),
}, },
}))); })));
} }
@ -685,6 +769,68 @@ export class NoteEntityService implements OnModuleInit {
}, {} as Record<string, string | undefined>); }, {} as Record<string, string | undefined>);
} }
private async getChannels(notes: MiNote[]): Promise<Map<string, MiChannel>> {
const channels = new Map<string, MiChannel>();
const channelsToFetch = new Set<string>();
for (const note of notes) {
if (note.channel) {
channels.set(note.channel.id, note.channel);
} else if (note.channelId) {
channelsToFetch.add(note.channelId);
}
}
if (channelsToFetch.size > 0) {
const newChannels = await this.channelsRepository.findBy({
id: In(Array.from(channelsToFetch)),
});
for (const channel of newChannels) {
channels.set(channel.id, channel);
}
}
return channels;
}
private async getReactions(notes: MiNote[], me: { id: string } | null | undefined) {
const bufferedReactions = this.meta.enableReactionsBuffering ? await this.reactionsBufferingService.getMany([...getAppearNoteIds(notes)]) : null;
const meId = me ? me.id : null;
const myReactionsMap = new Map<MiNote['id'], string | null>();
if (meId) {
const idsNeedFetchMyReaction = new Set<MiNote['id']>();
for (const note of notes) {
const reactionsCount = Object.values(this.reactionsBufferingService.mergeReactions(note.reactions, bufferedReactions?.get(note.id)?.deltas ?? {})).reduce((a, b) => a + b, 0);
if (reactionsCount === 0) {
myReactionsMap.set(note.id, null);
} else if (reactionsCount <= note.reactionAndUserPairCache.length + (bufferedReactions?.get(note.id)?.pairs.length ?? 0)) {
const pairInBuffer = bufferedReactions?.get(note.id)?.pairs.find(p => p[0] === meId);
if (pairInBuffer) {
myReactionsMap.set(note.id, pairInBuffer[1]);
} else {
const pair = note.reactionAndUserPairCache.find(p => p.startsWith(meId));
myReactionsMap.set(note.id, pair ? pair.split('/')[1] : null);
}
} else {
idsNeedFetchMyReaction.add(note.id);
}
}
const myReactions = idsNeedFetchMyReaction.size > 0 ? await this.noteReactionsRepository.findBy({
userId: meId,
noteId: In(Array.from(idsNeedFetchMyReaction)),
}) : [];
for (const id of idsNeedFetchMyReaction) {
myReactionsMap.set(id, myReactions.find(reaction => reaction.noteId === id)?.reaction ?? null);
}
}
return { bufferedReactions, myReactionsMap };
}
@bindThis @bindThis
public genLocalNoteUri(noteId: string): string { public genLocalNoteUri(noteId: string): string {
return `${this.config.url}/notes/${noteId}`; return `${this.config.url}/notes/${noteId}`;

View file

@ -30,6 +30,7 @@ import type {
FollowingsRepository, FollowingsRepository,
FollowRequestsRepository, FollowRequestsRepository,
MiFollowing, MiFollowing,
MiInstance,
MiMeta, MiMeta,
MiUserNotePining, MiUserNotePining,
MiUserProfile, MiUserProfile,
@ -42,7 +43,7 @@ import type {
UsersRepository, UsersRepository,
} from '@/models/_.js'; } from '@/models/_.js';
import { bindThis } from '@/decorators.js'; import { bindThis } from '@/decorators.js';
import { RoleService } from '@/core/RoleService.js'; import { RolePolicies, RoleService } from '@/core/RoleService.js';
import { ApPersonService } from '@/core/activitypub/models/ApPersonService.js'; import { ApPersonService } from '@/core/activitypub/models/ApPersonService.js';
import { FederatedInstanceService } from '@/core/FederatedInstanceService.js'; import { FederatedInstanceService } from '@/core/FederatedInstanceService.js';
import { IdService } from '@/core/IdService.js'; import { IdService } from '@/core/IdService.js';
@ -52,6 +53,7 @@ import { AvatarDecorationService } from '@/core/AvatarDecorationService.js';
import { ChatService } from '@/core/ChatService.js'; import { ChatService } from '@/core/ChatService.js';
import { isSystemAccount } from '@/misc/is-system-account.js'; import { isSystemAccount } from '@/misc/is-system-account.js';
import { DriveFileEntityService } from '@/core/entities/DriveFileEntityService.js'; import { DriveFileEntityService } from '@/core/entities/DriveFileEntityService.js';
import type { CacheService } from '@/core/CacheService.js';
import type { OnModuleInit } from '@nestjs/common'; import type { OnModuleInit } from '@nestjs/common';
import type { NoteEntityService } from './NoteEntityService.js'; import type { NoteEntityService } from './NoteEntityService.js';
import type { PageEntityService } from './PageEntityService.js'; import type { PageEntityService } from './PageEntityService.js';
@ -77,7 +79,7 @@ function isRemoteUser(user: MiUser | { host: MiUser['host'] }): boolean {
export type UserRelation = { export type UserRelation = {
id: MiUser['id'] id: MiUser['id']
following: MiFollowing | null, following: Omit<MiFollowing, 'isFollowerHibernated'> | null,
isFollowing: boolean isFollowing: boolean
isFollowed: boolean isFollowed: boolean
hasPendingFollowRequestFromYou: boolean hasPendingFollowRequestFromYou: boolean
@ -103,6 +105,7 @@ export class UserEntityService implements OnModuleInit {
private idService: IdService; private idService: IdService;
private avatarDecorationService: AvatarDecorationService; private avatarDecorationService: AvatarDecorationService;
private chatService: ChatService; private chatService: ChatService;
private cacheService: CacheService;
constructor( constructor(
private moduleRef: ModuleRef, private moduleRef: ModuleRef,
@ -163,6 +166,7 @@ export class UserEntityService implements OnModuleInit {
this.idService = this.moduleRef.get('IdService'); this.idService = this.moduleRef.get('IdService');
this.avatarDecorationService = this.moduleRef.get('AvatarDecorationService'); this.avatarDecorationService = this.moduleRef.get('AvatarDecorationService');
this.chatService = this.moduleRef.get('ChatService'); this.chatService = this.moduleRef.get('ChatService');
this.cacheService = this.moduleRef.get('CacheService');
} }
//#region Validators //#region Validators
@ -193,16 +197,8 @@ export class UserEntityService implements OnModuleInit {
memo, memo,
mutedInstances, mutedInstances,
] = await Promise.all([ ] = await Promise.all([
this.followingsRepository.findOneBy({ this.cacheService.userFollowingsCache.fetch(me).then(f => f.get(target) ?? null),
followerId: me, this.cacheService.userFollowingsCache.fetch(target).then(f => f.has(me)),
followeeId: target,
}),
this.followingsRepository.exists({
where: {
followerId: target,
followeeId: me,
},
}),
this.followRequestsRepository.exists({ this.followRequestsRepository.exists({
where: { where: {
followerId: me, followerId: me,
@ -215,45 +211,22 @@ export class UserEntityService implements OnModuleInit {
followeeId: me, followeeId: me,
}, },
}), }),
this.blockingsRepository.exists({ this.cacheService.userBlockingCache.fetch(me)
where: { .then(blockees => blockees.has(target)),
blockerId: me, this.cacheService.userBlockedCache.fetch(me)
blockeeId: target, .then(blockers => blockers.has(target)),
}, this.cacheService.userMutingsCache.fetch(me)
}), .then(mutings => mutings.has(target)),
this.blockingsRepository.exists({ this.cacheService.renoteMutingsCache.fetch(me)
where: { .then(mutings => mutings.has(target)),
blockerId: target, this.cacheService.findUserById(target).then(u => u.host),
blockeeId: me,
},
}),
this.mutingsRepository.exists({
where: {
muterId: me,
muteeId: target,
},
}),
this.renoteMutingsRepository.exists({
where: {
muterId: me,
muteeId: target,
},
}),
this.usersRepository.createQueryBuilder('u')
.select('u.host')
.where({ id: target })
.getRawOne<{ u_host: string }>()
.then(it => it?.u_host ?? null),
this.userMemosRepository.createQueryBuilder('m') this.userMemosRepository.createQueryBuilder('m')
.select('m.memo') .select('m.memo')
.where({ userId: me, targetUserId: target }) .where({ userId: me, targetUserId: target })
.getRawOne<{ m_memo: string | null }>() .getRawOne<{ m_memo: string | null }>()
.then(it => it?.m_memo ?? null), .then(it => it?.m_memo ?? null),
this.userProfilesRepository.createQueryBuilder('p') this.cacheService.userProfileCache.fetch(me)
.select('p.mutedInstances') .then(profile => profile.mutedInstances),
.where({ userId: me })
.getRawOne<{ p_mutedInstances: string[] }>()
.then(it => it?.p_mutedInstances ?? []),
]); ]);
const isInstanceMuted = !!host && mutedInstances.includes(host); const isInstanceMuted = !!host && mutedInstances.includes(host);
@ -277,8 +250,8 @@ export class UserEntityService implements OnModuleInit {
@bindThis @bindThis
public async getRelations(me: MiUser['id'], targets: MiUser['id'][]): Promise<Map<MiUser['id'], UserRelation>> { public async getRelations(me: MiUser['id'], targets: MiUser['id'][]): Promise<Map<MiUser['id'], UserRelation>> {
const [ const [
followers, myFollowing,
followees, myFollowers,
followersRequests, followersRequests,
followeesRequests, followeesRequests,
blockers, blockers,
@ -289,13 +262,8 @@ export class UserEntityService implements OnModuleInit {
memos, memos,
mutedInstances, mutedInstances,
] = await Promise.all([ ] = await Promise.all([
this.followingsRepository.findBy({ followerId: me }) this.cacheService.userFollowingsCache.fetch(me),
.then(f => new Map(f.map(it => [it.followeeId, it]))), this.cacheService.userFollowersCache.fetch(me),
this.followingsRepository.createQueryBuilder('f')
.select('f.followerId')
.where('f.followeeId = :me', { me })
.getRawMany<{ f_followerId: string }>()
.then(it => it.map(it => it.f_followerId)),
this.followRequestsRepository.createQueryBuilder('f') this.followRequestsRepository.createQueryBuilder('f')
.select('f.followeeId') .select('f.followeeId')
.where('f.followerId = :me', { me }) .where('f.followerId = :me', { me })
@ -306,34 +274,18 @@ export class UserEntityService implements OnModuleInit {
.where('f.followeeId = :me', { me }) .where('f.followeeId = :me', { me })
.getRawMany<{ f_followerId: string }>() .getRawMany<{ f_followerId: string }>()
.then(it => it.map(it => it.f_followerId)), .then(it => it.map(it => it.f_followerId)),
this.blockingsRepository.createQueryBuilder('b') this.cacheService.userBlockedCache.fetch(me),
.select('b.blockeeId') this.cacheService.userBlockingCache.fetch(me),
.where('b.blockerId = :me', { me }) this.cacheService.userMutingsCache.fetch(me),
.getRawMany<{ b_blockeeId: string }>() this.cacheService.renoteMutingsCache.fetch(me),
.then(it => it.map(it => it.b_blockeeId)), this.cacheService.getUsers(targets)
this.blockingsRepository.createQueryBuilder('b') .then(users => {
.select('b.blockerId') const record: Record<string, string | null> = {};
.where('b.blockeeId = :me', { me }) for (const [id, user] of users) {
.getRawMany<{ b_blockerId: string }>() record[id] = user.host;
.then(it => it.map(it => it.b_blockerId)), }
this.mutingsRepository.createQueryBuilder('m') return record;
.select('m.muteeId') }),
.where('m.muterId = :me', { me })
.getRawMany<{ m_muteeId: string }>()
.then(it => it.map(it => it.m_muteeId)),
this.renoteMutingsRepository.createQueryBuilder('m')
.select('m.muteeId')
.where('m.muterId = :me', { me })
.getRawMany<{ m_muteeId: string }>()
.then(it => it.map(it => it.m_muteeId)),
this.usersRepository.createQueryBuilder('u')
.select(['u.id', 'u.host'])
.where({ id: In(targets) } )
.getRawMany<{ m_id: string, m_host: string }>()
.then(it => it.reduce((map, it) => {
map[it.m_id] = it.m_host;
return map;
}, {} as Record<string, string>)),
this.userMemosRepository.createQueryBuilder('m') this.userMemosRepository.createQueryBuilder('m')
.select(['m.targetUserId', 'm.memo']) .select(['m.targetUserId', 'm.memo'])
.where({ userId: me, targetUserId: In(targets) }) .where({ userId: me, targetUserId: In(targets) })
@ -342,16 +294,13 @@ export class UserEntityService implements OnModuleInit {
map[it.m_targetUserId] = it.m_memo; map[it.m_targetUserId] = it.m_memo;
return map; return map;
}, {} as Record<string, string | null>)), }, {} as Record<string, string | null>)),
this.userProfilesRepository.createQueryBuilder('p') this.cacheService.userProfileCache.fetch(me)
.select('p.mutedInstances') .then(p => p.mutedInstances),
.where({ userId: me })
.getRawOne<{ p_mutedInstances: string[] }>()
.then(it => it?.p_mutedInstances ?? []),
]); ]);
return new Map( return new Map(
targets.map(target => { targets.map(target => {
const following = followers.get(target) ?? null; const following = myFollowing.get(target) ?? null;
return [ return [
target, target,
@ -359,14 +308,14 @@ export class UserEntityService implements OnModuleInit {
id: target, id: target,
following: following, following: following,
isFollowing: following != null, isFollowing: following != null,
isFollowed: followees.includes(target), isFollowed: myFollowers.has(target),
hasPendingFollowRequestFromYou: followersRequests.includes(target), hasPendingFollowRequestFromYou: followersRequests.includes(target),
hasPendingFollowRequestToYou: followeesRequests.includes(target), hasPendingFollowRequestToYou: followeesRequests.includes(target),
isBlocking: blockers.includes(target), isBlocking: blockees.has(target),
isBlocked: blockees.includes(target), isBlocked: blockers.has(target),
isMuted: muters.includes(target), isMuted: muters.has(target),
isRenoteMuted: renoteMuters.includes(target), isRenoteMuted: renoteMuters.has(target),
isInstanceMuted: mutedInstances.includes(hosts[target]), isInstanceMuted: hosts[target] != null && mutedInstances.includes(hosts[target]),
memo: memos[target] ?? null, memo: memos[target] ?? null,
}, },
]; ];
@ -391,6 +340,7 @@ export class UserEntityService implements OnModuleInit {
return false; // TODO return false; // TODO
} }
// TODO optimization: make redis calls in MULTI
@bindThis @bindThis
public async getNotificationsInfo(userId: MiUser['id']): Promise<{ public async getNotificationsInfo(userId: MiUser['id']): Promise<{
hasUnread: boolean; hasUnread: boolean;
@ -424,16 +374,14 @@ export class UserEntityService implements OnModuleInit {
@bindThis @bindThis
public async getHasPendingReceivedFollowRequest(userId: MiUser['id']): Promise<boolean> { public async getHasPendingReceivedFollowRequest(userId: MiUser['id']): Promise<boolean> {
const count = await this.followRequestsRepository.countBy({ return await this.followRequestsRepository.existsBy({
followeeId: userId, followeeId: userId,
}); });
return count > 0;
} }
@bindThis @bindThis
public async getHasPendingSentFollowRequest(userId: MiUser['id']): Promise<boolean> { public async getHasPendingSentFollowRequest(userId: MiUser['id']): Promise<boolean> {
return this.followRequestsRepository.existsBy({ return await this.followRequestsRepository.existsBy({
followerId: userId, followerId: userId,
}); });
} }
@ -480,6 +428,12 @@ export class UserEntityService implements OnModuleInit {
userRelations?: Map<MiUser['id'], UserRelation>, userRelations?: Map<MiUser['id'], UserRelation>,
userMemos?: Map<MiUser['id'], string | null>, userMemos?: Map<MiUser['id'], string | null>,
pinNotes?: Map<MiUser['id'], MiUserNotePining[]>, pinNotes?: Map<MiUser['id'], MiUserNotePining[]>,
iAmModerator?: boolean,
userIdsByUri?: Map<string, string>,
instances?: Map<string, MiInstance | null>,
securityKeyCounts?: Map<string, number>,
pendingReceivedFollows?: Set<string>,
pendingSentFollows?: Set<string>,
}, },
): Promise<Packed<S>> { ): Promise<Packed<S>> {
const opts = Object.assign({ const opts = Object.assign({
@ -521,7 +475,7 @@ export class UserEntityService implements OnModuleInit {
const isDetailed = opts.schema !== 'UserLite'; const isDetailed = opts.schema !== 'UserLite';
const meId = me ? me.id : null; const meId = me ? me.id : null;
const isMe = meId === user.id; const isMe = meId === user.id;
const iAmModerator = me ? await this.roleService.isModerator(me as MiUser) : false; const iAmModerator = opts.iAmModerator ?? (me ? await this.roleService.isModerator(me as MiUser) : false);
const profile = isDetailed const profile = isDetailed
? (opts.userProfile ?? user.userProfile ?? await this.userProfilesRepository.findOneByOrFail({ userId: user.id })) ? (opts.userProfile ?? user.userProfile ?? await this.userProfilesRepository.findOneByOrFail({ userId: user.id }))
@ -582,6 +536,9 @@ export class UserEntityService implements OnModuleInit {
const checkHost = user.host == null ? this.config.host : user.host; const checkHost = user.host == null ? this.config.host : user.host;
const notificationsInfo = isMe && isDetailed ? await this.getNotificationsInfo(user.id) : null; const notificationsInfo = isMe && isDetailed ? await this.getNotificationsInfo(user.id) : null;
let fetchPoliciesPromise: Promise<RolePolicies> | null = null;
const fetchPolicies = () => fetchPoliciesPromise ??= this.roleService.getUserPolicies(user);
const packed = { const packed = {
id: user.id, id: user.id,
name: user.name, name: user.name,
@ -607,13 +564,13 @@ export class UserEntityService implements OnModuleInit {
mandatoryCW: user.mandatoryCW, mandatoryCW: user.mandatoryCW,
rejectQuotes: user.rejectQuotes, rejectQuotes: user.rejectQuotes,
attributionDomains: user.attributionDomains, attributionDomains: user.attributionDomains,
isSilenced: user.isSilenced || this.roleService.getUserPolicies(user.id).then(r => !r.canPublicNote), isSilenced: user.isSilenced || fetchPolicies().then(r => !r.canPublicNote),
speakAsCat: user.speakAsCat ?? false, speakAsCat: user.speakAsCat ?? false,
approved: user.approved, approved: user.approved,
requireSigninToViewContents: user.requireSigninToViewContents === false ? undefined : true, requireSigninToViewContents: user.requireSigninToViewContents === false ? undefined : true,
makeNotesFollowersOnlyBefore: user.makeNotesFollowersOnlyBefore ?? undefined, makeNotesFollowersOnlyBefore: user.makeNotesFollowersOnlyBefore ?? undefined,
makeNotesHiddenBefore: user.makeNotesHiddenBefore ?? undefined, makeNotesHiddenBefore: user.makeNotesHiddenBefore ?? undefined,
instance: user.host ? this.federatedInstanceService.fetch(user.host).then(instance => instance ? { instance: user.host ? Promise.resolve(opts.instances?.has(user.host) ? opts.instances.get(user.host) : this.federatedInstanceService.fetch(user.host)).then(instance => instance ? {
name: instance.name, name: instance.name,
softwareName: instance.softwareName, softwareName: instance.softwareName,
softwareVersion: instance.softwareVersion, softwareVersion: instance.softwareVersion,
@ -628,7 +585,7 @@ export class UserEntityService implements OnModuleInit {
emojis: this.customEmojiService.populateEmojis(user.emojis, checkHost), emojis: this.customEmojiService.populateEmojis(user.emojis, checkHost),
onlineStatus: this.getOnlineStatus(user), onlineStatus: this.getOnlineStatus(user),
// パフォーマンス上の理由でローカルユーザーのみ // パフォーマンス上の理由でローカルユーザーのみ
badgeRoles: user.host == null ? this.roleService.getUserBadgeRoles(user.id).then((rs) => rs badgeRoles: user.host == null ? this.roleService.getUserBadgeRoles(user).then((rs) => rs
.filter((r) => r.isPublic || iAmModerator) .filter((r) => r.isPublic || iAmModerator)
.sort((a, b) => b.displayOrder - a.displayOrder) .sort((a, b) => b.displayOrder - a.displayOrder)
.map((r) => ({ .map((r) => ({
@ -641,9 +598,9 @@ export class UserEntityService implements OnModuleInit {
...(isDetailed ? { ...(isDetailed ? {
url: profile!.url, url: profile!.url,
uri: user.uri, uri: user.uri,
movedTo: user.movedToUri ? this.apPersonService.resolvePerson(user.movedToUri).then(user => user.id).catch(() => null) : null, movedTo: user.movedToUri ? Promise.resolve(opts.userIdsByUri?.get(user.movedToUri) ?? this.apPersonService.resolvePerson(user.movedToUri).then(user => user.id).catch(() => null)) : null,
alsoKnownAs: user.alsoKnownAs alsoKnownAs: user.alsoKnownAs
? Promise.all(user.alsoKnownAs.map(uri => this.apPersonService.fetchPerson(uri).then(user => user?.id).catch(() => null))) ? Promise.all(user.alsoKnownAs.map(uri => Promise.resolve(opts.userIdsByUri?.get(uri) ?? this.apPersonService.fetchPerson(uri).then(user => user?.id).catch(() => null))))
.then(xs => xs.length === 0 ? null : xs.filter(x => x != null)) .then(xs => xs.length === 0 ? null : xs.filter(x => x != null))
: null, : null,
updatedAt: user.updatedAt ? user.updatedAt.toISOString() : null, updatedAt: user.updatedAt ? user.updatedAt.toISOString() : null,
@ -670,8 +627,8 @@ export class UserEntityService implements OnModuleInit {
followersVisibility: profile!.followersVisibility, followersVisibility: profile!.followersVisibility,
followingVisibility: profile!.followingVisibility, followingVisibility: profile!.followingVisibility,
chatScope: user.chatScope, chatScope: user.chatScope,
canChat: this.roleService.getUserPolicies(user.id).then(r => r.chatAvailability === 'available'), canChat: fetchPolicies().then(r => r.chatAvailability === 'available'),
roles: this.roleService.getUserRoles(user.id).then(roles => roles.filter(role => role.isPublic).sort((a, b) => b.displayOrder - a.displayOrder).map(role => ({ roles: this.roleService.getUserRoles(user).then(roles => roles.filter(role => role.isPublic).sort((a, b) => b.displayOrder - a.displayOrder).map(role => ({
id: role.id, id: role.id,
name: role.name, name: role.name,
color: role.color, color: role.color,
@ -689,7 +646,7 @@ export class UserEntityService implements OnModuleInit {
twoFactorEnabled: profile!.twoFactorEnabled, twoFactorEnabled: profile!.twoFactorEnabled,
usePasswordLessLogin: profile!.usePasswordLessLogin, usePasswordLessLogin: profile!.usePasswordLessLogin,
securityKeys: profile!.twoFactorEnabled securityKeys: profile!.twoFactorEnabled
? this.userSecurityKeysRepository.countBy({ userId: user.id }).then(result => result >= 1) ? Promise.resolve(opts.securityKeyCounts?.get(user.id) ?? this.userSecurityKeysRepository.countBy({ userId: user.id })).then(result => result >= 1)
: false, : false,
} : {}), } : {}),
@ -722,8 +679,8 @@ export class UserEntityService implements OnModuleInit {
hasUnreadAntenna: this.getHasUnreadAntenna(user.id), hasUnreadAntenna: this.getHasUnreadAntenna(user.id),
hasUnreadChannel: false, // 後方互換性のため hasUnreadChannel: false, // 後方互換性のため
hasUnreadNotification: notificationsInfo?.hasUnread, // 後方互換性のため hasUnreadNotification: notificationsInfo?.hasUnread, // 後方互換性のため
hasPendingReceivedFollowRequest: this.getHasPendingReceivedFollowRequest(user.id), hasPendingReceivedFollowRequest: opts.pendingReceivedFollows?.has(user.id) ?? this.getHasPendingReceivedFollowRequest(user.id),
hasPendingSentFollowRequest: this.getHasPendingSentFollowRequest(user.id), hasPendingSentFollowRequest: opts.pendingSentFollows?.has(user.id) ?? this.getHasPendingSentFollowRequest(user.id),
unreadNotificationsCount: notificationsInfo?.unreadCount, unreadNotificationsCount: notificationsInfo?.unreadCount,
mutedWords: profile!.mutedWords, mutedWords: profile!.mutedWords,
hardMutedWords: profile!.hardMutedWords, hardMutedWords: profile!.hardMutedWords,
@ -733,7 +690,7 @@ export class UserEntityService implements OnModuleInit {
emailNotificationTypes: profile!.emailNotificationTypes, emailNotificationTypes: profile!.emailNotificationTypes,
achievements: profile!.achievements, achievements: profile!.achievements,
loggedInDays: profile!.loggedInDates.length, loggedInDays: profile!.loggedInDates.length,
policies: this.roleService.getUserPolicies(user.id), policies: fetchPolicies(),
defaultCW: profile!.defaultCW, defaultCW: profile!.defaultCW,
defaultCWPriority: profile!.defaultCWPriority, defaultCWPriority: profile!.defaultCWPriority,
allowUnsignedFetch: user.allowUnsignedFetch, allowUnsignedFetch: user.allowUnsignedFetch,
@ -783,6 +740,8 @@ export class UserEntityService implements OnModuleInit {
includeSecrets?: boolean, includeSecrets?: boolean,
}, },
): Promise<Packed<S>[]> { ): Promise<Packed<S>[]> {
if (users.length === 0) return [];
// -- IDのみの要素を補完して完全なエンティティ一覧を作る // -- IDのみの要素を補完して完全なエンティティ一覧を作る
const _users = users.filter((user): user is MiUser => typeof user !== 'string'); const _users = users.filter((user): user is MiUser => typeof user !== 'string');
@ -800,37 +759,44 @@ export class UserEntityService implements OnModuleInit {
} }
const _userIds = _users.map(u => u.id); const _userIds = _users.map(u => u.id);
// -- 実行者の有無や指定スキーマの種別によって要否が異なる値群を取得 const iAmModerator = await this.roleService.isModerator(me as MiUser);
const meId = me ? me.id : null;
const isMe = meId && _userIds.includes(meId);
const isDetailed = options && options.schema !== 'UserLite';
const isDetailedAndMe = isDetailed && isMe;
const isDetailedAndMeOrMod = isDetailed && (isMe || iAmModerator);
const isDetailedAndNotMe = isDetailed && !isMe;
let profilesMap: Map<MiUser['id'], MiUserProfile> = new Map(); const userUris = new Set(_users
let userRelations: Map<MiUser['id'], UserRelation> = new Map(); .flatMap(user => [user.uri, user.movedToUri])
let userMemos: Map<MiUser['id'], string | null> = new Map(); .filter((uri): uri is string => uri != null));
let pinNotes: Map<MiUser['id'], MiUserNotePining[]> = new Map();
if (options?.schema !== 'UserLite') { const userHosts = new Set(_users
const _profiles: MiUserProfile[] = []; .map(user => user.host)
.filter((host): host is string => host != null));
const _profilesFromUsers: [string, MiUserProfile][] = [];
const _profilesToFetch: string[] = []; const _profilesToFetch: string[] = [];
for (const user of _users) { for (const user of _users) {
if (user.userProfile) { if (user.userProfile) {
_profiles.push(user.userProfile); _profilesFromUsers.push([user.id, user.userProfile]);
} else { } else {
_profilesToFetch.push(user.id); _profilesToFetch.push(user.id);
} }
} }
if (_profilesToFetch.length > 0) {
const fetched = await this.userProfilesRepository.findBy({ userId: In(_profilesToFetch) });
_profiles.push(...fetched);
}
profilesMap = new Map(_profiles.map(p => [p.userId, p]));
const meId = me ? me.id : null; // -- 実行者の有無や指定スキーマの種別によって要否が異なる値群を取得
if (meId) {
userMemos = await this.userMemosRepository.findBy({ userId: meId })
.then(memos => new Map(memos.map(memo => [memo.targetUserId, memo.memo])));
if (_userIds.length > 0) { const [profilesMap, userMemos, userRelations, pinNotes, userIdsByUri, instances, securityKeyCounts, pendingReceivedFollows, pendingSentFollows] = await Promise.all([
userRelations = await this.getRelations(meId, _userIds); // profilesMap
pinNotes = await this.userNotePiningsRepository.createQueryBuilder('pin') this.cacheService.userProfileCache.fetchMany(_profilesToFetch).then(profiles => new Map(profiles.concat(_profilesFromUsers))),
// userMemos
isDetailed && meId ? this.userMemosRepository.findBy({ userId: meId })
.then(memos => new Map(memos.map(memo => [memo.targetUserId, memo.memo]))) : new Map(),
// userRelations
isDetailedAndNotMe && meId ? this.getRelations(meId, _userIds) : new Map(),
// pinNotes
isDetailed ? this.userNotePiningsRepository.createQueryBuilder('pin')
.where('pin.userId IN (:...userIds)', { userIds: _userIds }) .where('pin.userId IN (:...userIds)', { userIds: _userIds })
.innerJoinAndSelect('pin.note', 'note') .innerJoinAndSelect('pin.note', 'note')
.getMany() .getMany()
@ -846,10 +812,51 @@ export class UserEntityService implements OnModuleInit {
notes.sort((a, b) => b.id.localeCompare(a.id)); notes.sort((a, b) => b.id.localeCompare(a.id));
} }
return map; return map;
}); }) : new Map(),
} // userIdsByUrl
} isDetailed ? this.usersRepository.createQueryBuilder('user')
} .select([
'user.id',
'user.uri',
])
.where({
uri: In(Array.from(userUris)),
})
.getRawMany<{ user_uri: string, user_id: string }>()
.then(users => new Map(users.map(u => [u.user_uri, u.user_id]))) : new Map(),
// instances
Promise.all(Array.from(userHosts).map(async host => [host, await this.federatedInstanceService.fetch(host)] as const))
.then(hosts => new Map(hosts)),
// securityKeyCounts
isDetailedAndMeOrMod ? this.userSecurityKeysRepository.createQueryBuilder('key')
.select('key.userId', 'userId')
.addSelect('count(key.id)', 'userCount')
.where({
userId: In(_userIds),
})
.groupBy('key.userId')
.getRawMany<{ userId: string, userCount: number }>()
.then(counts => new Map(counts.map(c => [c.userId, c.userCount]))) : new Map(),
// TODO optimization: cache follow requests
// pendingReceivedFollows
isDetailedAndMe ? this.followRequestsRepository.createQueryBuilder('req')
.select('req.followeeId', 'followeeId')
.where({
followeeId: In(_userIds),
})
.groupBy('req.followeeId')
.getRawMany<{ followeeId: string }>()
.then(reqs => new Set(reqs.map(r => r.followeeId))) : new Set<string>(),
// pendingSentFollows
isDetailedAndMe ? this.followRequestsRepository.createQueryBuilder('req')
.select('req.followerId', 'followerId')
.where({
followerId: In(_userIds),
})
.groupBy('req.followerId')
.getRawMany<{ followerId: string }>()
.then(reqs => new Set(reqs.map(r => r.followerId))) : new Set<string>(),
]);
return Promise.all( return Promise.all(
_users.map(u => this.pack( _users.map(u => this.pack(
@ -861,6 +868,12 @@ export class UserEntityService implements OnModuleInit {
userRelations: userRelations, userRelations: userRelations,
userMemos: userMemos, userMemos: userMemos,
pinNotes: pinNotes, pinNotes: pinNotes,
iAmModerator,
userIdsByUri,
instances,
securityKeyCounts,
pendingReceivedFollows,
pendingSentFollows,
}, },
)), )),
); );

View file

@ -0,0 +1,385 @@
/*
* SPDX-FileCopyrightText: hazelnoot and other Sharkey contributors
* SPDX-License-Identifier: AGPL-3.0-only
*/
import { InternalEventService } from '@/core/InternalEventService.js';
import { bindThis } from '@/decorators.js';
import { InternalEventTypes } from '@/core/GlobalEventService.js';
import { MemoryKVCache } from '@/misc/cache.js';
export interface QuantumKVOpts<T> {
/**
* Memory cache lifetime in milliseconds.
*/
lifetime: number;
/**
* Callback to fetch the value for a key that wasn't found in the cache.
* May be synchronous or async.
*/
fetcher: (key: string, cache: QuantumKVCache<T>) => T | Promise<T>;
/**
* Optional callback to fetch the value for multiple keys that weren't found in the cache.
* May be synchronous or async.
* If not provided, then the implementation will fall back on repeated calls to fetcher().
*/
bulkFetcher?: (keys: string[], cache: QuantumKVCache<T>) => Iterable<[key: string, value: T]> | Promise<Iterable<[key: string, value: T]>>;
/**
* Optional callback when one or more values are changed (created, updated, or deleted) in the cache, either locally or elsewhere in the cluster.
* This is called *after* the cache state is updated.
* Implementations may be synchronous or async.
*/
onChanged?: (keys: string[], cache: QuantumKVCache<T>) => void | Promise<void>;
}
/**
* QuantumKVCache is a lifetime-bounded memory cache (like MemoryKVCache) with automatic cross-cluster synchronization via Redis.
* All nodes in the cluster are guaranteed to have a *subset* view of the current accurate state, though individual processes may have different items in their local cache.
* This ensures that a call to get() will never return stale data.
*/
export class QuantumKVCache<T> implements Iterable<[key: string, value: T]> {
private readonly memoryCache: MemoryKVCache<T>;
public readonly fetcher: QuantumKVOpts<T>['fetcher'];
public readonly bulkFetcher: QuantumKVOpts<T>['bulkFetcher'];
public readonly onChanged: QuantumKVOpts<T>['onChanged'];
/**
* @param internalEventService Service bus to synchronize events.
* @param name Unique name of the cache - must be the same in all processes.
* @param opts Cache options
*/
constructor(
private readonly internalEventService: InternalEventService,
private readonly name: string,
opts: QuantumKVOpts<T>,
) {
this.memoryCache = new MemoryKVCache(opts.lifetime);
this.fetcher = opts.fetcher;
this.bulkFetcher = opts.bulkFetcher;
this.onChanged = opts.onChanged;
this.internalEventService.on('quantumCacheUpdated', this.onQuantumCacheUpdated, {
// Ignore our own events, otherwise we'll immediately erase any set value.
ignoreLocal: true,
});
}
/**
* The number of items currently in memory.
* This applies to the local subset view, not the cross-cluster cache state.
*/
public get size() {
return this.memoryCache.size;
}
/**
* Iterates all [key, value] pairs in memory.
* This applies to the local subset view, not the cross-cluster cache state.
*/
@bindThis
public *entries(): Generator<[key: string, value: T]> {
for (const entry of this.memoryCache.entries) {
yield [entry[0], entry[1].value];
}
}
/**
* Iterates all keys in memory.
* This applies to the local subset view, not the cross-cluster cache state.
*/
@bindThis
public *keys() {
for (const entry of this.memoryCache.entries) {
yield entry[0];
}
}
/**
* Iterates all values pairs in memory.
* This applies to the local subset view, not the cross-cluster cache state.
*/
@bindThis
public *values() {
for (const entry of this.memoryCache.entries) {
yield entry[1].value;
}
}
/**
* Creates or updates a value in the cache, and erases any stale caches across the cluster.
* Fires an onSet event after the cache has been updated in all processes.
* Skips if the value is unchanged.
*/
@bindThis
public async set(key: string, value: T): Promise<void> {
if (this.memoryCache.get(key) === value) {
return;
}
this.memoryCache.set(key, value);
await this.internalEventService.emit('quantumCacheUpdated', { name: this.name, keys: [key] });
if (this.onChanged) {
await this.onChanged([key], this);
}
}
/**
* Creates or updates multiple value in the cache, and erases any stale caches across the cluster.
* Fires an onSet for each changed item event after the cache has been updated in all processes.
* Skips if all values are unchanged.
*/
@bindThis
public async setMany(items: Iterable<[key: string, value: T]>): Promise<void> {
const changedKeys: string[] = [];
for (const item of items) {
if (this.memoryCache.get(item[0]) !== item[1]) {
changedKeys.push(item[0]);
this.memoryCache.set(item[0], item[1]);
}
}
if (changedKeys.length > 0) {
await this.internalEventService.emit('quantumCacheUpdated', { name: this.name, keys: changedKeys });
if (this.onChanged) {
await this.onChanged(changedKeys, this);
}
}
}
/**
* Adds a value to the local memory cache without notifying other process.
* Neither a Redis event nor onSet callback will be fired, as the value has not actually changed.
* This should only be used when the value is known to be current, like after fetching from the database.
*/
@bindThis
public add(key: string, value: T): void {
this.memoryCache.set(key, value);
}
/**
* Adds multiple values to the local memory cache without notifying other process.
* Neither a Redis event nor onSet callback will be fired, as the value has not actually changed.
* This should only be used when the value is known to be current, like after fetching from the database.
*/
@bindThis
public addMany(items: Iterable<[key: string, value: T]>): void {
for (const [key, value] of items) {
this.memoryCache.set(key, value);
}
}
/**
* Gets a value from the local memory cache, or returns undefined if not found.
* Returns cached data only - does not make any fetches.
*/
@bindThis
public get(key: string): T | undefined {
return this.memoryCache.get(key);
}
/**
* Gets multiple values from the local memory cache; returning undefined for any missing keys.
* Returns cached data only - does not make any fetches.
*/
@bindThis
public getMany(keys: Iterable<string>): [key: string, value: T | undefined][] {
const results: [key: string, value: T | undefined][] = [];
for (const key of keys) {
results.push([key, this.get(key)]);
}
return results;
}
/**
* Gets or fetches a value from the cache.
* Fires an onSet event, but does not emit an update event to other processes.
*/
@bindThis
public async fetch(key: string): Promise<T> {
let value = this.memoryCache.get(key);
if (value === undefined) {
value = await this.fetcher(key, this);
this.memoryCache.set(key, value);
if (this.onChanged) {
await this.onChanged([key], this);
}
}
return value;
}
/**
* Gets or fetches multiple values from the cache.
* Fires onSet events, but does not emit any update events to other processes.
*/
@bindThis
public async fetchMany(keys: Iterable<string>): Promise<[key: string, value: T][]> {
const results: [key: string, value: T][] = [];
const toFetch: string[] = [];
// Spliterate into cached results / uncached keys.
for (const key of keys) {
const fromCache = this.get(key);
if (fromCache) {
results.push([key, fromCache]);
} else {
toFetch.push(key);
}
}
// Fetch any uncached keys
if (toFetch.length > 0) {
const fetched = await this.bulkFetch(toFetch);
// Add to cache and return set
this.addMany(fetched);
results.push(...fetched);
// Emit event
if (this.onChanged) {
await this.onChanged(toFetch, this);
}
}
return results;
}
/**
* Returns true is a key exists in memory.
* This applies to the local subset view, not the cross-cluster cache state.
*/
@bindThis
public has(key: string): boolean {
return this.memoryCache.get(key) !== undefined;
}
/**
* Deletes a value from the cache, and erases any stale caches across the cluster.
* Fires an onDelete event after the cache has been updated in all processes.
*/
@bindThis
public async delete(key: string): Promise<void> {
this.memoryCache.delete(key);
await this.internalEventService.emit('quantumCacheUpdated', { name: this.name, keys: [key] });
if (this.onChanged) {
await this.onChanged([key], this);
}
}
/**
* Deletes multiple values from the cache, and erases any stale caches across the cluster.
* Fires an onDelete event for each key after the cache has been updated in all processes.
* Skips if the input is empty.
*/
@bindThis
public async deleteMany(keys: Iterable<string>): Promise<void> {
const deleted: string[] = [];
for (const key of keys) {
this.memoryCache.delete(key);
deleted.push(key);
}
if (deleted.length === 0) {
return;
}
await this.internalEventService.emit('quantumCacheUpdated', { name: this.name, keys: deleted });
if (this.onChanged) {
await this.onChanged(deleted, this);
}
}
/**
* Refreshes the value of a key from the fetcher, and erases any stale caches across the cluster.
* Fires an onSet event after the cache has been updated in all processes.
*/
@bindThis
public async refresh(key: string): Promise<T> {
const value = await this.fetcher(key, this);
await this.set(key, value);
return value;
}
@bindThis
public async refreshMany(keys: Iterable<string>): Promise<[key: string, value: T][]> {
const values = await this.bulkFetch(keys);
await this.setMany(values);
return values;
}
/**
* Erases all entries from the local memory cache.
* Does not send any events or update other processes.
*/
@bindThis
public clear() {
this.memoryCache.clear();
}
/**
* Removes expired cache entries from the local view.
* Does not send any events or update other processes.
*/
@bindThis
public gc() {
this.memoryCache.gc();
}
/**
* Erases all data and disconnects from the cluster.
* This *must* be called when shutting down to prevent memory leaks!
*/
@bindThis
public dispose() {
this.internalEventService.off('quantumCacheUpdated', this.onQuantumCacheUpdated);
this.memoryCache.dispose();
}
@bindThis
private async bulkFetch(keys: Iterable<string>): Promise<[key: string, value: T][]> {
if (this.bulkFetcher) {
const results = await this.bulkFetcher(Array.from(keys), this);
return Array.from(results);
}
const results: [key: string, value: T][] = [];
for (const key of keys) {
const value = await this.fetcher(key, this);
results.push([key, value]);
}
return results;
}
@bindThis
private async onQuantumCacheUpdated(data: InternalEventTypes['quantumCacheUpdated']): Promise<void> {
if (data.name === this.name) {
for (const key of data.keys) {
this.memoryCache.delete(key);
}
if (this.onChanged) {
await this.onChanged(data.keys, this);
}
}
}
/**
* Iterates all [key, value] pairs in memory.
* This applies to the local subset view, not the cross-cluster cache state.
*/
[Symbol.iterator](): Iterator<[key: string, value: T]> {
return this.entries();
}
}

View file

@ -9,9 +9,9 @@ import { bindThis } from '@/decorators.js';
export class RedisKVCache<T> { export class RedisKVCache<T> {
private readonly lifetime: number; private readonly lifetime: number;
private readonly memoryCache: MemoryKVCache<T>; private readonly memoryCache: MemoryKVCache<T>;
private readonly fetcher: (key: string) => Promise<T>; public readonly fetcher: (key: string) => Promise<T>;
private readonly toRedisConverter: (value: T) => string; public readonly toRedisConverter: (value: T) => string;
private readonly fromRedisConverter: (value: string) => T | undefined; public readonly fromRedisConverter: (value: string) => T | undefined;
constructor( constructor(
private redisClient: Redis.Redis, private redisClient: Redis.Redis,
@ -99,6 +99,11 @@ export class RedisKVCache<T> {
// TODO: イベント発行して他プロセスのメモリキャッシュも更新できるようにする // TODO: イベント発行して他プロセスのメモリキャッシュも更新できるようにする
} }
@bindThis
public clear() {
this.memoryCache.clear();
}
@bindThis @bindThis
public gc() { public gc() {
this.memoryCache.gc(); this.memoryCache.gc();
@ -113,9 +118,9 @@ export class RedisKVCache<T> {
export class RedisSingleCache<T> { export class RedisSingleCache<T> {
private readonly lifetime: number; private readonly lifetime: number;
private readonly memoryCache: MemorySingleCache<T>; private readonly memoryCache: MemorySingleCache<T>;
private readonly fetcher: () => Promise<T>; public readonly fetcher: () => Promise<T>;
private readonly toRedisConverter: (value: T) => string; public readonly toRedisConverter: (value: T) => string;
private readonly fromRedisConverter: (value: string) => T | undefined; public readonly fromRedisConverter: (value: string) => T | undefined;
constructor( constructor(
private redisClient: Redis.Redis, private redisClient: Redis.Redis,
@ -123,16 +128,17 @@ export class RedisSingleCache<T> {
opts: { opts: {
lifetime: number; lifetime: number;
memoryCacheLifetime: number; memoryCacheLifetime: number;
fetcher: RedisSingleCache<T>['fetcher']; fetcher?: RedisSingleCache<T>['fetcher'];
toRedisConverter: RedisSingleCache<T>['toRedisConverter']; toRedisConverter?: RedisSingleCache<T>['toRedisConverter'];
fromRedisConverter: RedisSingleCache<T>['fromRedisConverter']; fromRedisConverter?: RedisSingleCache<T>['fromRedisConverter'];
}, },
) { ) {
this.lifetime = opts.lifetime; this.lifetime = opts.lifetime;
this.memoryCache = new MemorySingleCache(opts.memoryCacheLifetime); this.memoryCache = new MemorySingleCache(opts.memoryCacheLifetime);
this.fetcher = opts.fetcher;
this.toRedisConverter = opts.toRedisConverter; this.fetcher = opts.fetcher ?? (() => { throw new Error('fetch not supported - use get/set directly'); });
this.fromRedisConverter = opts.fromRedisConverter; this.toRedisConverter = opts.toRedisConverter ?? ((value) => JSON.stringify(value));
this.fromRedisConverter = opts.fromRedisConverter ?? ((value) => JSON.parse(value));
} }
@bindThis @bindThis
@ -237,6 +243,16 @@ export class MemoryKVCache<T> {
return cached.value; return cached.value;
} }
public has(key: string): boolean {
const cached = this.cache.get(key);
if (cached == null) return false;
if ((Date.now() - cached.date) > this.lifetime) {
this.cache.delete(key);
return false;
}
return true;
}
@bindThis @bindThis
public delete(key: string): void { public delete(key: string): void {
this.cache.delete(key); this.cache.delete(key);
@ -322,6 +338,10 @@ export class MemoryKVCache<T> {
clearInterval(this.gcIntervalHandle); clearInterval(this.gcIntervalHandle);
} }
public get size() {
return this.cache.size;
}
public get entries() { public get entries() {
return this.cache.entries(); return this.cache.entries();
} }

View file

@ -18,6 +18,7 @@ import { SearchService } from '@/core/SearchService.js';
import { ApLogService } from '@/core/ApLogService.js'; import { ApLogService } from '@/core/ApLogService.js';
import { ReactionService } from '@/core/ReactionService.js'; import { ReactionService } from '@/core/ReactionService.js';
import { QueueService } from '@/core/QueueService.js'; import { QueueService } from '@/core/QueueService.js';
import { CacheService } from '@/core/CacheService.js';
import { QueueLoggerService } from '../QueueLoggerService.js'; import { QueueLoggerService } from '../QueueLoggerService.js';
import type * as Bull from 'bullmq'; import type * as Bull from 'bullmq';
import type { DbUserDeleteJobData } from '../types.js'; import type { DbUserDeleteJobData } from '../types.js';
@ -94,6 +95,7 @@ export class DeleteAccountProcessorService {
private searchService: SearchService, private searchService: SearchService,
private reactionService: ReactionService, private reactionService: ReactionService,
private readonly apLogService: ApLogService, private readonly apLogService: ApLogService,
private readonly cacheService: CacheService,
) { ) {
this.logger = this.queueLoggerService.logger.createSubLogger('delete-account'); this.logger = this.queueLoggerService.logger.createSubLogger('delete-account');
} }
@ -140,6 +142,22 @@ export class DeleteAccountProcessorService {
} }
{ // Delete user relations { // Delete user relations
await this.cacheService.refreshFollowRelationsFor(user.id);
await this.cacheService.userFollowingsCache.delete(user.id);
await this.cacheService.userFollowingsCache.delete(user.id);
await this.cacheService.userBlockingCache.delete(user.id);
await this.cacheService.userBlockedCache.delete(user.id);
await this.cacheService.userMutingsCache.delete(user.id);
await this.cacheService.userMutingsCache.delete(user.id);
await this.cacheService.hibernatedUserCache.delete(user.id);
await this.cacheService.renoteMutingsCache.delete(user.id);
await this.cacheService.userProfileCache.delete(user.id);
this.cacheService.userByIdCache.delete(user.id);
this.cacheService.localUserByIdCache.delete(user.id);
if (user.token) {
this.cacheService.localUserByNativeTokenCache.delete(user.token);
}
await this.followingsRepository.delete({ await this.followingsRepository.delete({
followerId: user.id, followerId: user.id,
}); });

View file

@ -47,7 +47,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
alwaysMarkNsfw: true, alwaysMarkNsfw: true,
}); });
await this.cacheService.userProfileCache.refresh(ps.userId); await this.cacheService.userProfileCache.delete(ps.userId);
}); });
} }
} }

View file

@ -12,6 +12,7 @@ import { UserFollowingService } from '@/core/UserFollowingService.js';
import { DI } from '@/di-symbols.js'; import { DI } from '@/di-symbols.js';
import { GetterService } from '@/server/api/GetterService.js'; import { GetterService } from '@/server/api/GetterService.js';
import { ApiError } from '../../error.js'; import { ApiError } from '../../error.js';
import { CacheService } from '@/core/CacheService.js';
export const meta = { export const meta = {
tags: ['following', 'users'], tags: ['following', 'users'],
@ -69,6 +70,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
private userEntityService: UserEntityService, private userEntityService: UserEntityService,
private getterService: GetterService, private getterService: GetterService,
private userFollowingService: UserFollowingService, private userFollowingService: UserFollowingService,
private readonly cacheService: CacheService,
) { ) {
super(meta, paramDef, async (ps, me) => { super(meta, paramDef, async (ps, me) => {
const follower = me; const follower = me;
@ -85,12 +87,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
}); });
// Check not following // Check not following
const exist = await this.followingsRepository.exists({ const exist = await this.cacheService.userFollowingsCache.fetch(follower.id).then(f => f.has(followee.id));
where: {
followerId: follower.id,
followeeId: followee.id,
},
});
if (!exist) { if (!exist) {
throw new ApiError(meta.errors.notFollowing); throw new ApiError(meta.errors.notFollowing);

View file

@ -11,6 +11,7 @@ import { UserEntityService } from '@/core/entities/UserEntityService.js';
import { UserFollowingService } from '@/core/UserFollowingService.js'; import { UserFollowingService } from '@/core/UserFollowingService.js';
import { DI } from '@/di-symbols.js'; import { DI } from '@/di-symbols.js';
import { GetterService } from '@/server/api/GetterService.js'; import { GetterService } from '@/server/api/GetterService.js';
import { CacheService } from '@/core/CacheService.js';
import { ApiError } from '../../error.js'; import { ApiError } from '../../error.js';
export const meta = { export const meta = {
@ -69,6 +70,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
private userEntityService: UserEntityService, private userEntityService: UserEntityService,
private getterService: GetterService, private getterService: GetterService,
private userFollowingService: UserFollowingService, private userFollowingService: UserFollowingService,
private readonly cacheService: CacheService,
) { ) {
super(meta, paramDef, async (ps, me) => { super(meta, paramDef, async (ps, me) => {
const followee = me; const followee = me;
@ -85,12 +87,9 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
}); });
// Check not following // Check not following
const exist = await this.followingsRepository.findOneBy({ const isFollowing = await this.cacheService.userFollowingsCache.fetch(follower.id).then(f => f.has(followee.id));
followerId: follower.id,
followeeId: followee.id,
});
if (exist == null) { if (!isFollowing) {
throw new ApiError(meta.errors.notFollowing); throw new ApiError(meta.errors.notFollowing);
} }

View file

@ -12,6 +12,7 @@ import { UserFollowingService } from '@/core/UserFollowingService.js';
import { DI } from '@/di-symbols.js'; import { DI } from '@/di-symbols.js';
import { GetterService } from '@/server/api/GetterService.js'; import { GetterService } from '@/server/api/GetterService.js';
import { ApiError } from '../../error.js'; import { ApiError } from '../../error.js';
import { CacheService } from '@/core/CacheService.js';
export const meta = { export const meta = {
tags: ['following', 'users'], tags: ['following', 'users'],
@ -39,6 +40,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
constructor( constructor(
@Inject(DI.followingsRepository) @Inject(DI.followingsRepository)
private followingsRepository: FollowingsRepository, private followingsRepository: FollowingsRepository,
private readonly cacheService: CacheService,
) { ) {
super(meta, paramDef, async (ps, me) => { super(meta, paramDef, async (ps, me) => {
await this.followingsRepository.update({ await this.followingsRepository.update({
@ -48,6 +50,8 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
withReplies: ps.withReplies != null ? ps.withReplies : undefined, withReplies: ps.withReplies != null ? ps.withReplies : undefined,
}); });
await this.cacheService.refreshFollowRelationsFor(me.id);
return; return;
}); });
} }

View file

@ -11,6 +11,7 @@ import { UserEntityService } from '@/core/entities/UserEntityService.js';
import { UserFollowingService } from '@/core/UserFollowingService.js'; import { UserFollowingService } from '@/core/UserFollowingService.js';
import { DI } from '@/di-symbols.js'; import { DI } from '@/di-symbols.js';
import { GetterService } from '@/server/api/GetterService.js'; import { GetterService } from '@/server/api/GetterService.js';
import { CacheService } from '@/core/CacheService.js';
import { ApiError } from '../../error.js'; import { ApiError } from '../../error.js';
export const meta = { export const meta = {
@ -71,6 +72,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
private userEntityService: UserEntityService, private userEntityService: UserEntityService,
private getterService: GetterService, private getterService: GetterService,
private userFollowingService: UserFollowingService, private userFollowingService: UserFollowingService,
private readonly cacheService: CacheService,
) { ) {
super(meta, paramDef, async (ps, me) => { super(meta, paramDef, async (ps, me) => {
const follower = me; const follower = me;
@ -87,10 +89,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
}); });
// Check not following // Check not following
const exist = await this.followingsRepository.findOneBy({ const exist = await this.cacheService.userFollowingsCache.fetch(follower.id).then(f => f.get(followee.id));
followerId: follower.id,
followeeId: followee.id,
});
if (exist == null) { if (exist == null) {
throw new ApiError(meta.errors.notFollowing); throw new ApiError(meta.errors.notFollowing);
@ -103,6 +102,8 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
withReplies: ps.withReplies != null ? ps.withReplies : undefined, withReplies: ps.withReplies != null ? ps.withReplies : undefined,
}); });
await this.cacheService.refreshFollowRelationsFor(follower.id);
return await this.userEntityService.pack(follower.id, me); return await this.userEntityService.pack(follower.id, me);
}); });
} }

View file

@ -617,7 +617,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
const updatedProfile = await this.userProfilesRepository.findOneByOrFail({ userId: user.id }); const updatedProfile = await this.userProfilesRepository.findOneByOrFail({ userId: user.id });
this.cacheService.userProfileCache.set(user.id, updatedProfile); await this.cacheService.userProfileCache.set(user.id, updatedProfile);
// Publish meUpdated event // Publish meUpdated event
this.globalEventService.publishMainStream(user.id, 'meUpdated', iObj); this.globalEventService.publishMainStream(user.id, 'meUpdated', iObj);

View file

@ -348,7 +348,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
throw new ApiError(meta.errors.noSuchReplyTarget); throw new ApiError(meta.errors.noSuchReplyTarget);
} else if (isRenote(reply) && !isQuote(reply)) { } else if (isRenote(reply) && !isQuote(reply)) {
throw new ApiError(meta.errors.cannotReplyToPureRenote); throw new ApiError(meta.errors.cannotReplyToPureRenote);
} else if (!await this.noteEntityService.isVisibleForMe(reply, me.id)) { } else if (!await this.noteEntityService.isVisibleForMe(reply, me.id, { me })) {
throw new ApiError(meta.errors.cannotReplyToInvisibleNote); throw new ApiError(meta.errors.cannotReplyToInvisibleNote);
} else if (reply.visibility === 'specified' && ps.visibility !== 'specified') { } else if (reply.visibility === 'specified' && ps.visibility !== 'specified') {
throw new ApiError(meta.errors.cannotReplyToSpecifiedVisibilityNoteWithExtendedVisibility); throw new ApiError(meta.errors.cannotReplyToSpecifiedVisibilityNoteWithExtendedVisibility);

View file

@ -402,7 +402,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
throw new ApiError(meta.errors.noSuchReplyTarget); throw new ApiError(meta.errors.noSuchReplyTarget);
} else if (isRenote(reply) && !isQuote(reply)) { } else if (isRenote(reply) && !isQuote(reply)) {
throw new ApiError(meta.errors.cannotReplyToPureRenote); throw new ApiError(meta.errors.cannotReplyToPureRenote);
} else if (!await this.noteEntityService.isVisibleForMe(reply, me.id)) { } else if (!await this.noteEntityService.isVisibleForMe(reply, me.id, { me })) {
throw new ApiError(meta.errors.cannotReplyToInvisibleNote); throw new ApiError(meta.errors.cannotReplyToInvisibleNote);
} else if (reply.visibility === 'specified' && ps.visibility !== 'specified') { } else if (reply.visibility === 'specified' && ps.visibility !== 'specified') {
throw new ApiError(meta.errors.cannotReplyToSpecifiedVisibilityNoteWithExtendedVisibility); throw new ApiError(meta.errors.cannotReplyToSpecifiedVisibilityNoteWithExtendedVisibility);

View file

@ -164,7 +164,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
excludeBots: !ps.withBots, excludeBots: !ps.withBots,
noteFilter: note => { noteFilter: note => {
if (note.reply && note.reply.visibility === 'followers') { if (note.reply && note.reply.visibility === 'followers') {
if (!Object.hasOwn(followings, note.reply.userId) && note.reply.userId !== me.id) return false; if (!followings.has(note.reply.userId) && note.reply.userId !== me.id) return false;
} }
return true; return true;

View file

@ -115,7 +115,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
excludePureRenotes: !ps.withRenotes, excludePureRenotes: !ps.withRenotes,
noteFilter: note => { noteFilter: note => {
if (note.reply && note.reply.visibility === 'followers') { if (note.reply && note.reply.visibility === 'followers') {
if (!Object.hasOwn(followings, note.reply.userId) && note.reply.userId !== me.id) return false; if (!followings.has(note.reply.userId) && note.reply.userId !== me.id) return false;
} }
if (!ps.withBots && note.user?.isBot) return false; if (!ps.withBots && note.user?.isBot) return false;

View file

@ -91,7 +91,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
throw err; throw err;
}); });
if (!(await this.noteEntityService.isVisibleForMe(note, me?.id ?? null))) { if (!(await this.noteEntityService.isVisibleForMe(note, me?.id ?? null, { me }))) {
throw new ApiError(meta.errors.cannotTranslateInvisibleNote); throw new ApiError(meta.errors.cannotTranslateInvisibleNote);
} }

View file

@ -104,7 +104,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
sendReadMessage: ps.sendReadMessage, sendReadMessage: ps.sendReadMessage,
}); });
this.pushNotificationService.refreshCache(me.id); await this.pushNotificationService.refreshCache(me.id);
return { return {
state: 'subscribed' as const, state: 'subscribed' as const,

View file

@ -46,7 +46,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
}); });
if (me) { if (me) {
this.pushNotificationService.refreshCache(me.id); await this.pushNotificationService.refreshCache(me.id);
} }
}); });
} }

View file

@ -86,7 +86,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
sendReadMessage: swSubscription.sendReadMessage, sendReadMessage: swSubscription.sendReadMessage,
}); });
this.pushNotificationService.refreshCache(me.id); await this.pushNotificationService.refreshCache(me.id);
return { return {
userId: swSubscription.userId, userId: swSubscription.userId,

View file

@ -12,6 +12,7 @@ import { FollowingEntityService } from '@/core/entities/FollowingEntityService.j
import { UtilityService } from '@/core/UtilityService.js'; import { UtilityService } from '@/core/UtilityService.js';
import { DI } from '@/di-symbols.js'; import { DI } from '@/di-symbols.js';
import { RoleService } from '@/core/RoleService.js'; import { RoleService } from '@/core/RoleService.js';
import { CacheService } from '@/core/CacheService.js';
import { ApiError } from '../../error.js'; import { ApiError } from '../../error.js';
export const meta = { export const meta = {
@ -89,6 +90,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
private followingEntityService: FollowingEntityService, private followingEntityService: FollowingEntityService,
private queryService: QueryService, private queryService: QueryService,
private roleService: RoleService, private roleService: RoleService,
private readonly cacheService: CacheService,
) { ) {
super(meta, paramDef, async (ps, me) => { super(meta, paramDef, async (ps, me) => {
const user = await this.usersRepository.findOneBy(ps.userId != null const user = await this.usersRepository.findOneBy(ps.userId != null
@ -110,12 +112,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
if (me == null) { if (me == null) {
throw new ApiError(meta.errors.forbidden); throw new ApiError(meta.errors.forbidden);
} else if (me.id !== user.id) { } else if (me.id !== user.id) {
const isFollowing = await this.followingsRepository.exists({ const isFollowing = await this.cacheService.userFollowingsCache.fetch(me.id).then(f => f.has(user.id));
where: {
followeeId: user.id,
followerId: me.id,
},
});
if (!isFollowing) { if (!isFollowing) {
throw new ApiError(meta.errors.forbidden); throw new ApiError(meta.errors.forbidden);
} }

View file

@ -13,6 +13,7 @@ import { FollowingEntityService } from '@/core/entities/FollowingEntityService.j
import { UtilityService } from '@/core/UtilityService.js'; import { UtilityService } from '@/core/UtilityService.js';
import { DI } from '@/di-symbols.js'; import { DI } from '@/di-symbols.js';
import { RoleService } from '@/core/RoleService.js'; import { RoleService } from '@/core/RoleService.js';
import { CacheService } from '@/core/CacheService.js';
import { ApiError } from '../../error.js'; import { ApiError } from '../../error.js';
export const meta = { export const meta = {
@ -98,6 +99,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
private followingEntityService: FollowingEntityService, private followingEntityService: FollowingEntityService,
private queryService: QueryService, private queryService: QueryService,
private roleService: RoleService, private roleService: RoleService,
private readonly cacheService: CacheService,
) { ) {
super(meta, paramDef, async (ps, me) => { super(meta, paramDef, async (ps, me) => {
const user = await this.usersRepository.findOneBy(ps.userId != null const user = await this.usersRepository.findOneBy(ps.userId != null
@ -119,12 +121,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
if (me == null) { if (me == null) {
throw new ApiError(meta.errors.forbidden); throw new ApiError(meta.errors.forbidden);
} else if (me.id !== user.id) { } else if (me.id !== user.id) {
const isFollowing = await this.followingsRepository.exists({ const isFollowing = await this.cacheService.userFollowingsCache.fetch(me.id).then(f => f.has(user.id));
where: {
followeeId: user.id,
followerId: me.id,
},
});
if (!isFollowing) { if (!isFollowing) {
throw new ApiError(meta.errors.forbidden); throw new ApiError(meta.errors.forbidden);
} }

View file

@ -134,7 +134,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
if (ps.withReplies) redisTimelines.push(`userTimelineWithReplies:${ps.userId}`); if (ps.withReplies) redisTimelines.push(`userTimelineWithReplies:${ps.userId}`);
if (ps.withChannelNotes) redisTimelines.push(`userTimelineWithChannel:${ps.userId}`); if (ps.withChannelNotes) redisTimelines.push(`userTimelineWithChannel:${ps.userId}`);
const isFollowing = me && Object.hasOwn(await this.cacheService.userFollowingsCache.fetch(me.id), ps.userId); const isFollowing = me && (await this.cacheService.userFollowingsCache.fetch(me.id)).has(ps.userId);
const timeline = await this.fanoutTimelineEndpointService.timeline({ const timeline = await this.fanoutTimelineEndpointService.timeline({
untilId, untilId,

View file

@ -71,6 +71,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
this.queryService.generateBlockQueryForUsers(query, me); this.queryService.generateBlockQueryForUsers(query, me);
this.queryService.generateBlockedUserQueryForNotes(query, me); this.queryService.generateBlockedUserQueryForNotes(query, me);
// TODO optimization: replace with exists()
const followingQuery = this.followingsRepository.createQueryBuilder('following') const followingQuery = this.followingsRepository.createQueryBuilder('following')
.select('following.followeeId') .select('following.followeeId')
.where('following.followerId = :followerId', { followerId: me.id }); .where('following.followerId = :followerId', { followerId: me.id });

View file

@ -57,19 +57,19 @@ export class MastodonDataService {
if (relations.reply) { if (relations.reply) {
query.leftJoinAndSelect('note.reply', 'reply'); query.leftJoinAndSelect('note.reply', 'reply');
if (typeof(relations.reply) === 'object') { if (typeof(relations.reply) === 'object') {
if (relations.reply.reply) query.leftJoinAndSelect('note.reply.reply', 'replyReply'); if (relations.reply.reply) query.leftJoinAndSelect('reply.reply', 'replyReply');
if (relations.reply.renote) query.leftJoinAndSelect('note.reply.renote', 'replyRenote'); if (relations.reply.renote) query.leftJoinAndSelect('reply.renote', 'replyRenote');
if (relations.reply.user) query.innerJoinAndSelect('note.reply.user', 'replyUser'); if (relations.reply.user) query.innerJoinAndSelect('reply.user', 'replyUser');
if (relations.reply.channel) query.leftJoinAndSelect('note.reply.channel', 'replyChannel'); if (relations.reply.channel) query.leftJoinAndSelect('reply.channel', 'replyChannel');
} }
} }
if (relations.renote) { if (relations.renote) {
query.leftJoinAndSelect('note.renote', 'renote'); query.leftJoinAndSelect('note.renote', 'renote');
if (typeof(relations.renote) === 'object') { if (typeof(relations.renote) === 'object') {
if (relations.renote.reply) query.leftJoinAndSelect('note.renote.reply', 'renoteReply'); if (relations.renote.reply) query.leftJoinAndSelect('renote.reply', 'renoteReply');
if (relations.renote.renote) query.leftJoinAndSelect('note.renote.renote', 'renoteRenote'); if (relations.renote.renote) query.leftJoinAndSelect('renote.renote', 'renoteRenote');
if (relations.renote.user) query.innerJoinAndSelect('note.renote.user', 'renoteUser'); if (relations.renote.user) query.innerJoinAndSelect('renote.user', 'renoteUser');
if (relations.renote.channel) query.leftJoinAndSelect('note.renote.channel', 'renoteChannel'); if (relations.renote.channel) query.leftJoinAndSelect('renote.channel', 'renoteChannel');
} }
} }
if (relations.user) { if (relations.user) {

View file

@ -36,7 +36,7 @@ export default class Connection {
private channels = new Map<string, Channel>(); private channels = new Map<string, Channel>();
private subscribingNotes = new Map<string, number>(); private subscribingNotes = new Map<string, number>();
public userProfile: MiUserProfile | null = null; public userProfile: MiUserProfile | null = null;
public following: Record<string, Pick<MiFollowing, 'withReplies'> | undefined> = {}; public following: Map<string, Omit<MiFollowing, 'isFollowerHibernated'>> = new Map();
public followingChannels: Set<string> = new Set(); public followingChannels: Set<string> = new Set();
public userIdsWhoMeMuting: Set<string> = new Set(); public userIdsWhoMeMuting: Set<string> = new Set();
public userIdsWhoBlockingMe: Set<string> = new Set(); public userIdsWhoBlockingMe: Set<string> = new Set();

View file

@ -70,7 +70,7 @@ export default abstract class Channel {
if (!this.user) return false; if (!this.user) return false;
if (this.user.id === note.userId) return true; if (this.user.id === note.userId) return true;
if (note.visibility === 'followers') { if (note.visibility === 'followers') {
return this.following[note.userId] != null; return this.following.has(note.userId);
} }
if (!note.visibleUserIds) return false; if (!note.visibleUserIds) return false;
return note.visibleUserIds.includes(this.user.id); return note.visibleUserIds.includes(this.user.id);
@ -84,7 +84,7 @@ export default abstract class Channel {
if (note.user.requireSigninToViewContents && !this.user) return true; if (note.user.requireSigninToViewContents && !this.user) return true;
// 流れてきたNoteがインスタンスミュートしたインスタンスが関わる // 流れてきたNoteがインスタンスミュートしたインスタンスが関わる
if (isInstanceMuted(note, this.userMutedInstances) && !this.following[note.userId]) return true; if (isInstanceMuted(note, this.userMutedInstances) && !this.following.has(note.userId)) return true;
// 流れてきたNoteがミュートしているユーザーが関わる // 流れてきたNoteがミュートしているユーザーが関わる
if (isUserRelated(note, this.userIdsWhoMeMuting)) return true; if (isUserRelated(note, this.userIdsWhoMeMuting)) return true;
@ -101,7 +101,7 @@ export default abstract class Channel {
if (note.user.isSilenced || note.user.instance?.isSilenced) { if (note.user.isSilenced || note.user.instance?.isSilenced) {
if (this.user == null) return true; if (this.user == null) return true;
if (this.user.id === note.userId) return false; if (this.user.id === note.userId) return false;
if (this.following[note.userId] == null) return true; if (!this.following.has(note.userId)) return true;
} }
// TODO muted threads // TODO muted threads

View file

@ -62,7 +62,7 @@ class BubbleTimelineChannel extends Channel {
const reply = note.reply; const reply = note.reply;
// 自分のフォローしていないユーザーの visibility: followers な投稿への返信は弾く // 自分のフォローしていないユーザーの visibility: followers な投稿への返信は弾く
if (!this.isNoteVisibleToMe(reply)) return; if (!this.isNoteVisibleToMe(reply)) return;
if (!this.following[note.userId]?.withReplies) { if (!this.following.get(note.userId)?.withReplies) {
// 「チャンネル接続主への返信」でもなければ、「チャンネル接続主が行った返信」でもなければ、「投稿者の投稿者自身への返信」でもない場合 // 「チャンネル接続主への返信」でもなければ、「チャンネル接続主が行った返信」でもなければ、「投稿者の投稿者自身への返信」でもない場合
if (reply.userId !== this.user?.id && !isMe && reply.userId !== note.userId) return; if (reply.userId !== this.user?.id && !isMe && reply.userId !== note.userId) return;
} }

View file

@ -63,7 +63,7 @@ class GlobalTimelineChannel extends Channel {
const reply = note.reply; const reply = note.reply;
// 自分のフォローしていないユーザーの visibility: followers な投稿への返信は弾く // 自分のフォローしていないユーザーの visibility: followers な投稿への返信は弾く
if (!this.isNoteVisibleToMe(reply)) return; if (!this.isNoteVisibleToMe(reply)) return;
if (!this.following[note.userId]?.withReplies) { if (!this.following.get(note.userId)?.withReplies) {
// 「チャンネル接続主への返信」でもなければ、「チャンネル接続主が行った返信」でもなければ、「投稿者の投稿者自身への返信」でもない場合 // 「チャンネル接続主への返信」でもなければ、「チャンネル接続主が行った返信」でもなければ、「投稿者の投稿者自身への返信」でもない場合
if (reply.userId !== this.user?.id && !isMe && reply.userId !== note.userId) return; if (reply.userId !== this.user?.id && !isMe && reply.userId !== note.userId) return;
} }

View file

@ -47,7 +47,7 @@ class HomeTimelineChannel extends Channel {
if (!this.followingChannels.has(note.channelId)) return; if (!this.followingChannels.has(note.channelId)) return;
} else { } else {
// その投稿のユーザーをフォローしていなかったら弾く // その投稿のユーザーをフォローしていなかったら弾く
if (!isMe && !Object.hasOwn(this.following, note.userId)) return; if (!isMe && !this.following.has(note.userId)) return;
} }
if (this.isNoteMutedOrBlocked(note)) return; if (this.isNoteMutedOrBlocked(note)) return;
@ -57,7 +57,7 @@ class HomeTimelineChannel extends Channel {
const reply = note.reply; const reply = note.reply;
// 自分のフォローしていないユーザーの visibility: followers な投稿への返信は弾く // 自分のフォローしていないユーザーの visibility: followers な投稿への返信は弾く
if (!this.isNoteVisibleToMe(reply)) return; if (!this.isNoteVisibleToMe(reply)) return;
if (!this.following[note.userId]?.withReplies) { if (!this.following.get(note.userId)?.withReplies) {
// 「チャンネル接続主への返信」でもなければ、「チャンネル接続主が行った返信」でもなければ、「投稿者の投稿者自身への返信」でもない場合 // 「チャンネル接続主への返信」でもなければ、「チャンネル接続主が行った返信」でもなければ、「投稿者の投稿者自身への返信」でもない場合
if (reply.userId !== this.user!.id && !isMe && reply.userId !== note.userId) return; if (reply.userId !== this.user!.id && !isMe && reply.userId !== note.userId) return;
} }

View file

@ -62,7 +62,7 @@ class HybridTimelineChannel extends Channel {
// フォローしているチャンネルの投稿 の場合だけ // フォローしているチャンネルの投稿 の場合だけ
if (!( if (!(
(note.channelId == null && isMe) || (note.channelId == null && isMe) ||
(note.channelId == null && Object.hasOwn(this.following, note.userId)) || (note.channelId == null && this.following.has(note.userId)) ||
(note.channelId == null && (note.user.host == null && note.visibility === 'public')) || (note.channelId == null && (note.user.host == null && note.visibility === 'public')) ||
(note.channelId != null && this.followingChannels.has(note.channelId)) (note.channelId != null && this.followingChannels.has(note.channelId))
)) return; )) return;
@ -74,7 +74,7 @@ class HybridTimelineChannel extends Channel {
const reply = note.reply; const reply = note.reply;
// 自分のフォローしていないユーザーの visibility: followers な投稿への返信は弾く // 自分のフォローしていないユーザーの visibility: followers な投稿への返信は弾く
if (!this.isNoteVisibleToMe(reply)) return; if (!this.isNoteVisibleToMe(reply)) return;
if (!this.following[note.userId]?.withReplies && !this.withReplies) { if (!this.following.get(note.userId)?.withReplies && !this.withReplies) {
// 「チャンネル接続主への返信」でもなければ、「チャンネル接続主が行った返信」でもなければ、「投稿者の投稿者自身への返信」でもない場合 // 「チャンネル接続主への返信」でもなければ、「チャンネル接続主が行った返信」でもなければ、「投稿者の投稿者自身への返信」でもない場合
if (reply.userId !== this.user!.id && !isMe && reply.userId !== note.userId) return; if (reply.userId !== this.user!.id && !isMe && reply.userId !== note.userId) return;
} }

View file

@ -67,7 +67,7 @@ class LocalTimelineChannel extends Channel {
const reply = note.reply; const reply = note.reply;
// 自分のフォローしていないユーザーの visibility: followers な投稿への返信は弾く // 自分のフォローしていないユーザーの visibility: followers な投稿への返信は弾く
if (!this.isNoteVisibleToMe(reply)) return; if (!this.isNoteVisibleToMe(reply)) return;
if (!this.following[note.userId]?.withReplies) { if (!this.following.get(note.userId)?.withReplies) {
// 「チャンネル接続主への返信」でもなければ、「チャンネル接続主が行った返信」でもなければ、「投稿者の投稿者自身への返信」でもない場合 // 「チャンネル接続主への返信」でもなければ、「チャンネル接続主が行った返信」でもなければ、「投稿者の投稿者自身への返信」でもない場合
if (reply.userId !== this.user?.id && !isMe && reply.userId !== note.userId) return; if (reply.userId !== this.user?.id && !isMe && reply.userId !== note.userId) return;
} }

View file

@ -55,7 +55,7 @@ class RoleTimelineChannel extends Channel {
const reply = note.reply; const reply = note.reply;
// 自分のフォローしていないユーザーの visibility: followers な投稿への返信は弾く // 自分のフォローしていないユーザーの visibility: followers な投稿への返信は弾く
if (!this.isNoteVisibleToMe(reply)) return; if (!this.isNoteVisibleToMe(reply)) return;
if (!this.following[note.userId]?.withReplies) { if (!this.following.get(note.userId)?.withReplies) {
// 「チャンネル接続主への返信」でもなければ、「チャンネル接続主が行った返信」でもなければ、「投稿者の投稿者自身への返信」でもない場合 // 「チャンネル接続主への返信」でもなければ、「チャンネル接続主が行った返信」でもなければ、「投稿者の投稿者自身への返信」でもない場合
if (reply.userId !== this.user?.id && !isMe && reply.userId !== note.userId) return; if (reply.userId !== this.user?.id && !isMe && reply.userId !== note.userId) return;
} }

View file

@ -98,7 +98,7 @@ class UserListChannel extends Channel {
const reply = note.reply; const reply = note.reply;
// 自分のフォローしていないユーザーの visibility: followers な投稿への返信は弾く // 自分のフォローしていないユーザーの visibility: followers な投稿への返信は弾く
if (!this.isNoteVisibleToMe(reply)) return; if (!this.isNoteVisibleToMe(reply)) return;
if (!this.following[note.userId]?.withReplies) { if (!this.following.get(note.userId)?.withReplies) {
// 「チャンネル接続主への返信」でもなければ、「チャンネル接続主が行った返信」でもなければ、「投稿者の投稿者自身への返信」でもない場合 // 「チャンネル接続主への返信」でもなければ、「チャンネル接続主が行った返信」でもなければ、「投稿者の投稿者自身への返信」でもない場合
if (reply.userId !== this.user!.id && !isMe && reply.userId !== note.userId) return; if (reply.userId !== this.user!.id && !isMe && reply.userId !== note.userId) return;
} }

View file

@ -0,0 +1,92 @@
/*
* SPDX-FileCopyrightText: hazelnoot and other Sharkey contributors
* SPDX-License-Identifier: AGPL-3.0-only
*/
import type { Listener, ListenerProps } from '@/core/InternalEventService.js';
import type Redis from 'ioredis';
import type { GlobalEventService, InternalEventTypes } from '@/core/GlobalEventService.js';
import { InternalEventService } from '@/core/InternalEventService.js';
import { bindThis } from '@/decorators.js';
type FakeCall<K extends keyof InternalEventService> = [K, Parameters<InternalEventService[K]>];
type FakeListener<K extends keyof InternalEventTypes> = [K, Listener<K>, ListenerProps];
/**
* Minimal implementation of InternalEventService meant for use in unit tests.
* There is no redis connection, and metadata is tracked in the public _calls and _listeners arrays.
* The on/off/emit methods are fully functional and can be called in tests to invoke any registered listeners.
*/
export class FakeInternalEventService extends InternalEventService {
/**
* List of calls to public methods, in chronological order.
*/
public _calls: FakeCall<keyof InternalEventService>[] = [];
/**
* List of currently registered listeners.
*/
public _listeners: FakeListener<keyof InternalEventTypes>[] = [];
/**
* Resets the mock.
* Clears all listeners and tracked calls.
*/
public _reset() {
this._calls = [];
this._listeners = [];
}
/**
* Simulates a remote event sent from another process in the cluster via redis.
*/
@bindThis
public async _emitRedis<K extends keyof InternalEventTypes>(type: K, value: InternalEventTypes[K]): Promise<void> {
await this.emit(type, value, false);
}
constructor() {
super(
{ on: () => {} } as unknown as Redis.Redis,
{} as unknown as GlobalEventService,
);
}
@bindThis
public on<K extends keyof InternalEventTypes>(type: K, listener: Listener<K>, props?: ListenerProps): void {
if (!this._listeners.some(l => l[0] === type && l[1] === listener)) {
this._listeners.push([type, listener as Listener<keyof InternalEventTypes>, props ?? {}]);
}
this._calls.push(['on', [type, listener as Listener<keyof InternalEventTypes>, props]]);
}
@bindThis
public off<K extends keyof InternalEventTypes>(type: K, listener: Listener<K>): void {
this._listeners = this._listeners.filter(l => l[0] !== type || l[1] !== listener);
this._calls.push(['off', [type, listener as Listener<keyof InternalEventTypes>]]);
}
@bindThis
public async emit<K extends keyof InternalEventTypes>(type: K, value: InternalEventTypes[K], isLocal = true): Promise<void> {
for (const listener of this._listeners) {
if (listener[0] === type) {
if ((isLocal && !listener[2].ignoreLocal) || (!isLocal && !listener[2].ignoreRemote)) {
await listener[1](value, type, isLocal);
}
}
}
this._calls.push(['emit', [type, value]]);
}
@bindThis
public dispose(): void {
this._listeners = [];
this._calls.push(['dispose', []]);
}
@bindThis
public onApplicationShutdown(): void {
this._calls.push(['onApplicationShutdown', []]);
}
}

View file

@ -0,0 +1,187 @@
/*
* SPDX-FileCopyrightText: hazelnoot and other Sharkey contributors
* SPDX-License-Identifier: AGPL-3.0-only
*/
import * as Redis from 'ioredis';
import { Inject } from '@nestjs/common';
import { FakeInternalEventService } from './FakeInternalEventService.js';
import type { BlockingsRepository, FollowingsRepository, MiUser, MutingsRepository, RenoteMutingsRepository, UserProfilesRepository, UsersRepository } from '@/models/_.js';
import type { MiLocalUser } from '@/models/User.js';
import { MemoryKVCache, MemorySingleCache, RedisKVCache, RedisSingleCache } from '@/misc/cache.js';
import { QuantumKVCache, QuantumKVOpts } from '@/misc/QuantumKVCache.js';
import { CacheService, FollowStats } from '@/core/CacheService.js';
import { DI } from '@/di-symbols.js';
import { UserEntityService } from '@/core/entities/UserEntityService.js';
import { InternalEventService } from '@/core/InternalEventService.js';
export function noOpRedis() {
return {
set: () => Promise.resolve(),
get: () => Promise.resolve(null),
del: () => Promise.resolve(),
on: () => {},
off: () => {},
} as unknown as Redis.Redis;
}
export class NoOpCacheService extends CacheService {
public readonly fakeRedis: {
[K in keyof Redis.Redis]: Redis.Redis[K];
};
public readonly fakeInternalEventService: FakeInternalEventService;
constructor(
@Inject(DI.usersRepository)
usersRepository: UsersRepository,
@Inject(DI.userProfilesRepository)
userProfilesRepository: UserProfilesRepository,
@Inject(DI.mutingsRepository)
mutingsRepository: MutingsRepository,
@Inject(DI.blockingsRepository)
blockingsRepository: BlockingsRepository,
@Inject(DI.renoteMutingsRepository)
renoteMutingsRepository: RenoteMutingsRepository,
@Inject(DI.followingsRepository)
followingsRepository: FollowingsRepository,
@Inject(UserEntityService)
userEntityService: UserEntityService,
) {
const fakeRedis = noOpRedis();
const fakeInternalEventService = new FakeInternalEventService();
super(
fakeRedis,
fakeRedis,
usersRepository,
userProfilesRepository,
mutingsRepository,
blockingsRepository,
renoteMutingsRepository,
followingsRepository,
userEntityService,
fakeInternalEventService,
);
this.fakeRedis = fakeRedis;
this.fakeInternalEventService = fakeInternalEventService;
// Override caches
this.userByIdCache = new NoOpMemoryKVCache<MiUser>();
this.localUserByNativeTokenCache = new NoOpMemoryKVCache<MiLocalUser | null>();
this.localUserByIdCache = new NoOpMemoryKVCache<MiLocalUser>();
this.uriPersonCache = new NoOpMemoryKVCache<MiUser | null>();
this.userProfileCache = NoOpQuantumKVCache.copy(this.userProfileCache, fakeInternalEventService);
this.userMutingsCache = NoOpQuantumKVCache.copy(this.userMutingsCache, fakeInternalEventService);
this.userBlockingCache = NoOpQuantumKVCache.copy(this.userBlockingCache, fakeInternalEventService);
this.userBlockedCache = NoOpQuantumKVCache.copy(this.userBlockedCache, fakeInternalEventService);
this.renoteMutingsCache = NoOpQuantumKVCache.copy(this.renoteMutingsCache, fakeInternalEventService);
this.userFollowingsCache = NoOpQuantumKVCache.copy(this.userFollowingsCache, fakeInternalEventService);
this.userFollowersCache = NoOpQuantumKVCache.copy(this.userFollowersCache, fakeInternalEventService);
this.hibernatedUserCache = NoOpQuantumKVCache.copy(this.hibernatedUserCache, fakeInternalEventService);
this.userFollowStatsCache = new NoOpMemoryKVCache<FollowStats>();
this.translationsCache = NoOpRedisKVCache.copy(this.translationsCache, fakeRedis);
}
}
export class NoOpMemoryKVCache<T> extends MemoryKVCache<T> {
constructor() {
super(-1);
}
}
export class NoOpMemorySingleCache<T> extends MemorySingleCache<T> {
constructor() {
super(-1);
}
}
export class NoOpRedisKVCache<T> extends RedisKVCache<T> {
constructor(opts?: {
redis?: Redis.Redis;
fetcher?: RedisKVCache<T>['fetcher'];
toRedisConverter?: RedisKVCache<T>['toRedisConverter'];
fromRedisConverter?: RedisKVCache<T>['fromRedisConverter'];
}) {
super(
opts?.redis ?? noOpRedis(),
'no-op',
{
lifetime: -1,
memoryCacheLifetime: -1,
fetcher: opts?.fetcher,
toRedisConverter: opts?.toRedisConverter,
fromRedisConverter: opts?.fromRedisConverter,
},
);
}
public static copy<T>(cache: RedisKVCache<T>, redis?: Redis.Redis): NoOpRedisKVCache<T> {
return new NoOpRedisKVCache<T>({
redis,
fetcher: cache.fetcher,
toRedisConverter: cache.toRedisConverter,
fromRedisConverter: cache.fromRedisConverter,
});
}
}
export class NoOpRedisSingleCache<T> extends RedisSingleCache<T> {
constructor(opts?: {
redis?: Redis.Redis;
fetcher?: RedisSingleCache<T>['fetcher'];
toRedisConverter?: RedisSingleCache<T>['toRedisConverter'];
fromRedisConverter?: RedisSingleCache<T>['fromRedisConverter'];
}) {
super(
opts?.redis ?? noOpRedis(),
'no-op',
{
lifetime: -1,
memoryCacheLifetime: -1,
fetcher: opts?.fetcher,
toRedisConverter: opts?.toRedisConverter,
fromRedisConverter: opts?.fromRedisConverter,
},
);
}
public static copy<T>(cache: RedisSingleCache<T>, redis?: Redis.Redis): NoOpRedisSingleCache<T> {
return new NoOpRedisSingleCache<T>({
redis,
fetcher: cache.fetcher,
toRedisConverter: cache.toRedisConverter,
fromRedisConverter: cache.fromRedisConverter,
});
}
}
export class NoOpQuantumKVCache<T> extends QuantumKVCache<T> {
constructor(opts: Omit<QuantumKVOpts<T>, 'lifetime'> & {
internalEventService?: InternalEventService,
}) {
super(
opts.internalEventService ?? new FakeInternalEventService(),
'no-op',
{
...opts,
lifetime: -1,
},
);
}
public static copy<T>(cache: QuantumKVCache<T>, internalEventService?: InternalEventService): NoOpQuantumKVCache<T> {
return new NoOpQuantumKVCache<T>({
internalEventService,
fetcher: cache.fetcher,
bulkFetcher: cache.bulkFetcher,
onChanged: cache.onChanged,
});
}
}

View file

@ -8,9 +8,12 @@ process.env.NODE_ENV = 'test';
import { jest } from '@jest/globals'; import { jest } from '@jest/globals';
import { ModuleMocker } from 'jest-mock'; import { ModuleMocker } from 'jest-mock';
import { Test } from '@nestjs/testing'; import { Test } from '@nestjs/testing';
import { NoOpCacheService } from '../misc/noOpCaches.js';
import { FakeInternalEventService } from '../misc/FakeInternalEventService.js';
import { GlobalModule } from '@/GlobalModule.js'; import { GlobalModule } from '@/GlobalModule.js';
import { AnnouncementService } from '@/core/AnnouncementService.js'; import { AnnouncementService } from '@/core/AnnouncementService.js';
import { AnnouncementEntityService } from '@/core/entities/AnnouncementEntityService.js'; import { AnnouncementEntityService } from '@/core/entities/AnnouncementEntityService.js';
import { InternalEventService } from '@/core/InternalEventService.js';
import type { import type {
AnnouncementReadsRepository, AnnouncementReadsRepository,
AnnouncementsRepository, AnnouncementsRepository,
@ -71,24 +74,27 @@ describe('AnnouncementService', () => {
AnnouncementEntityService, AnnouncementEntityService,
CacheService, CacheService,
IdService, IdService,
InternalEventService,
GlobalEventService,
ModerationLogService,
], ],
}) })
.useMocker((token) => { .useMocker((token) => {
if (token === GlobalEventService) { if (typeof token === 'function') {
return {
publishMainStream: jest.fn(),
publishBroadcastStream: jest.fn(),
};
} else if (token === ModerationLogService) {
return {
log: jest.fn(),
};
} else if (typeof token === 'function') {
const mockMetadata = moduleMocker.getMetadata(token) as MockFunctionMetadata<any, any>; const mockMetadata = moduleMocker.getMetadata(token) as MockFunctionMetadata<any, any>;
const Mock = moduleMocker.generateFromMetadata(mockMetadata); const Mock = moduleMocker.generateFromMetadata(mockMetadata);
return new Mock(); return new Mock();
} }
}) })
.overrideProvider(GlobalEventService).useValue({
publishMainStream: jest.fn(),
publishBroadcastStream: jest.fn(),
} as unknown as GlobalEventService)
.overrideProvider(ModerationLogService).useValue({
log: jest.fn(),
})
.overrideProvider(InternalEventService).useClass(FakeInternalEventService)
.overrideProvider(CacheService).useClass(NoOpCacheService)
.compile(); .compile();
app.enableShutdownHooks(); app.enableShutdownHooks();

View file

@ -10,12 +10,15 @@ import { jest } from '@jest/globals';
import { ModuleMocker } from 'jest-mock'; import { ModuleMocker } from 'jest-mock';
import { Test } from '@nestjs/testing'; import { Test } from '@nestjs/testing';
import * as lolex from '@sinonjs/fake-timers'; import * as lolex from '@sinonjs/fake-timers';
import { NoOpCacheService } from '../misc/noOpCaches.js';
import { FakeInternalEventService } from '../misc/FakeInternalEventService.js';
import type { TestingModule } from '@nestjs/testing'; import type { TestingModule } from '@nestjs/testing';
import type { MockFunctionMetadata } from 'jest-mock'; import type { MockFunctionMetadata } from 'jest-mock';
import { GlobalModule } from '@/GlobalModule.js'; import { GlobalModule } from '@/GlobalModule.js';
import { RoleService } from '@/core/RoleService.js'; import { RoleService } from '@/core/RoleService.js';
import { import {
InstancesRepository, InstancesRepository,
MetasRepository,
MiMeta, MiMeta,
MiRole, MiRole,
MiRoleAssignment, MiRoleAssignment,
@ -34,6 +37,7 @@ import { secureRndstr } from '@/misc/secure-rndstr.js';
import { NotificationService } from '@/core/NotificationService.js'; import { NotificationService } from '@/core/NotificationService.js';
import { RoleCondFormulaValue } from '@/models/Role.js'; import { RoleCondFormulaValue } from '@/models/Role.js';
import { UserEntityService } from '@/core/entities/UserEntityService.js'; import { UserEntityService } from '@/core/entities/UserEntityService.js';
import { InternalEventService } from '@/core/InternalEventService.js';
const moduleMocker = new ModuleMocker(global); const moduleMocker = new ModuleMocker(global);
@ -45,6 +49,7 @@ describe('RoleService', () => {
let rolesRepository: RolesRepository; let rolesRepository: RolesRepository;
let roleAssignmentsRepository: RoleAssignmentsRepository; let roleAssignmentsRepository: RoleAssignmentsRepository;
let meta: jest.Mocked<MiMeta>; let meta: jest.Mocked<MiMeta>;
let metasRepository: MetasRepository;
let notificationService: jest.Mocked<NotificationService>; let notificationService: jest.Mocked<NotificationService>;
let clock: lolex.InstalledClock; let clock: lolex.InstalledClock;
@ -143,18 +148,20 @@ describe('RoleService', () => {
provide: NotificationService.name, provide: NotificationService.name,
useExisting: NotificationService, useExisting: NotificationService,
}, },
MetaService,
InternalEventService,
], ],
}) })
.useMocker((token) => { .useMocker((token) => {
if (token === MetaService) {
return { fetch: jest.fn() };
}
if (typeof token === 'function') { if (typeof token === 'function') {
const mockMetadata = moduleMocker.getMetadata(token) as MockFunctionMetadata<any, any>; const mockMetadata = moduleMocker.getMetadata(token) as MockFunctionMetadata<any, any>;
const Mock = moduleMocker.generateFromMetadata(mockMetadata); const Mock = moduleMocker.generateFromMetadata(mockMetadata);
return new Mock(); return new Mock();
} }
}) })
.overrideProvider(MetaService).useValue({ fetch: jest.fn() })
.overrideProvider(InternalEventService).useClass(FakeInternalEventService)
.overrideProvider(CacheService).useClass(NoOpCacheService)
.compile(); .compile();
app.enableShutdownHooks(); app.enableShutdownHooks();
@ -164,6 +171,7 @@ describe('RoleService', () => {
usersRepository = app.get<UsersRepository>(DI.usersRepository); usersRepository = app.get<UsersRepository>(DI.usersRepository);
rolesRepository = app.get<RolesRepository>(DI.rolesRepository); rolesRepository = app.get<RolesRepository>(DI.rolesRepository);
roleAssignmentsRepository = app.get<RoleAssignmentsRepository>(DI.roleAssignmentsRepository); roleAssignmentsRepository = app.get<RoleAssignmentsRepository>(DI.roleAssignmentsRepository);
metasRepository = app.get<MetasRepository>(DI.metasRepository);
meta = app.get<MiMeta>(DI.meta) as jest.Mocked<MiMeta>; meta = app.get<MiMeta>(DI.meta) as jest.Mocked<MiMeta>;
notificationService = app.get<NotificationService>(NotificationService) as jest.Mocked<NotificationService>; notificationService = app.get<NotificationService>(NotificationService) as jest.Mocked<NotificationService>;
@ -175,7 +183,7 @@ describe('RoleService', () => {
clock.uninstall(); clock.uninstall();
await Promise.all([ await Promise.all([
app.get(DI.metasRepository).delete({}), metasRepository.delete({}),
usersRepository.delete({}), usersRepository.delete({}),
rolesRepository.delete({}), rolesRepository.delete({}),
roleAssignmentsRepository.delete({}), roleAssignmentsRepository.delete({}),

View file

@ -9,8 +9,12 @@ import { generateKeyPair } from 'crypto';
import { Test } from '@nestjs/testing'; import { Test } from '@nestjs/testing';
import { jest } from '@jest/globals'; import { jest } from '@jest/globals';
import { NoOpCacheService } from '../misc/noOpCaches.js';
import { FakeInternalEventService } from '../misc/FakeInternalEventService.js';
import type { Config } from '@/config.js'; import type { Config } from '@/config.js';
import type { MiLocalUser, MiRemoteUser } from '@/models/User.js'; import type { MiLocalUser, MiRemoteUser } from '@/models/User.js';
import { InternalEventService } from '@/core/InternalEventService.js';
import { CacheService } from '@/core/CacheService.js';
import { ApImageService } from '@/core/activitypub/models/ApImageService.js'; import { ApImageService } from '@/core/activitypub/models/ApImageService.js';
import { ApNoteService } from '@/core/activitypub/models/ApNoteService.js'; import { ApNoteService } from '@/core/activitypub/models/ApNoteService.js';
import { ApPersonService } from '@/core/activitypub/models/ApPersonService.js'; import { ApPersonService } from '@/core/activitypub/models/ApPersonService.js';
@ -30,7 +34,7 @@ import { genAidx } from '@/misc/id/aidx.js';
import { IdService } from '@/core/IdService.js'; import { IdService } from '@/core/IdService.js';
import { MockResolver } from '../misc/mock-resolver.js'; import { MockResolver } from '../misc/mock-resolver.js';
import { UserKeypairService } from '@/core/UserKeypairService.js'; import { UserKeypairService } from '@/core/UserKeypairService.js';
import { MemoryKVCache, RedisKVCache } from '@/misc/cache.js'; import { MemoryKVCache } from '@/misc/cache.js';
const host = 'https://host1.test'; const host = 'https://host1.test';
@ -154,6 +158,8 @@ describe('ActivityPub', () => {
}, },
}) })
.overrideProvider(DI.meta).useFactory({ factory: () => meta }) .overrideProvider(DI.meta).useFactory({ factory: () => meta })
.overrideProvider(CacheService).useClass(NoOpCacheService)
.overrideProvider(InternalEventService).useClass(FakeInternalEventService)
.compile(); .compile();
await app.init(); await app.init();
@ -556,7 +562,7 @@ describe('ActivityPub', () => {
publicKey, publicKey,
privateKey, privateKey,
}); });
((userKeypairService as unknown as { cache: RedisKVCache<MiUserKeypair> }).cache as unknown as { memoryCache: MemoryKVCache<MiUserKeypair> }).memoryCache.set(author.id, keypair); (userKeypairService as unknown as { cache: MemoryKVCache<MiUserKeypair> }).cache.set(author.id, keypair);
note = new MiNote({ note = new MiNote({
id: idService.gen(), id: idService.gen(),

View file

@ -4,6 +4,8 @@
*/ */
import { Test, TestingModule } from '@nestjs/testing'; import { Test, TestingModule } from '@nestjs/testing';
import { FakeInternalEventService } from '../../misc/FakeInternalEventService.js';
import { NoOpCacheService } from '../../misc/noOpCaches.js';
import type { MiUser } from '@/models/User.js'; import type { MiUser } from '@/models/User.js';
import { UserEntityService } from '@/core/entities/UserEntityService.js'; import { UserEntityService } from '@/core/entities/UserEntityService.js';
import { GlobalModule } from '@/GlobalModule.js'; import { GlobalModule } from '@/GlobalModule.js';
@ -51,6 +53,7 @@ import { ReactionService } from '@/core/ReactionService.js';
import { NotificationService } from '@/core/NotificationService.js'; import { NotificationService } from '@/core/NotificationService.js';
import { ReactionsBufferingService } from '@/core/ReactionsBufferingService.js'; import { ReactionsBufferingService } from '@/core/ReactionsBufferingService.js';
import { ChatService } from '@/core/ChatService.js'; import { ChatService } from '@/core/ChatService.js';
import { InternalEventService } from '@/core/InternalEventService.js';
process.env.NODE_ENV = 'test'; process.env.NODE_ENV = 'test';
@ -174,6 +177,7 @@ describe('UserEntityService', () => {
ReactionsBufferingService, ReactionsBufferingService,
NotificationService, NotificationService,
ChatService, ChatService,
InternalEventService,
]; ];
app = await Test.createTestingModule({ app = await Test.createTestingModule({
@ -182,7 +186,10 @@ describe('UserEntityService', () => {
...services, ...services,
...services.map(x => ({ provide: x.name, useExisting: x })), ...services.map(x => ({ provide: x.name, useExisting: x })),
], ],
}).compile(); })
.overrideProvider(InternalEventService).useClass(FakeInternalEventService)
.overrideProvider(CacheService).useClass(NoOpCacheService)
.compile();
await app.init(); await app.init();
app.enableShutdownHooks(); app.enableShutdownHooks();

View file

@ -0,0 +1,799 @@
/*
* SPDX-FileCopyrightText: hazelnoot and other Sharkey contributors
* SPDX-License-Identifier: AGPL-3.0-only
*/
import { jest } from '@jest/globals';
import { FakeInternalEventService } from '../../misc/FakeInternalEventService.js';
import { QuantumKVCache, QuantumKVOpts } from '@/misc/QuantumKVCache.js';
describe(QuantumKVCache, () => {
let fakeInternalEventService: FakeInternalEventService;
let madeCaches: { dispose: () => void }[];
function makeCache<T>(opts?: Partial<QuantumKVOpts<T>> & { name?: string }): QuantumKVCache<T> {
const _opts = {
name: 'test',
lifetime: Infinity,
fetcher: () => { throw new Error('not implemented'); },
} satisfies QuantumKVOpts<T> & { name: string };
if (opts) {
Object.assign(_opts, opts);
}
const cache = new QuantumKVCache<T>(fakeInternalEventService, _opts.name, _opts);
madeCaches.push(cache);
return cache;
}
beforeEach(() => {
madeCaches = [];
fakeInternalEventService = new FakeInternalEventService();
});
afterEach(() => {
madeCaches.forEach(cache => {
cache.dispose();
});
});
it('should connect on construct', () => {
makeCache();
expect(fakeInternalEventService._calls).toContainEqual(['on', ['quantumCacheUpdated', expect.anything(), { ignoreLocal: true }]]);
});
it('should disconnect on dispose', () => {
const cache = makeCache();
cache.dispose();
const callback = fakeInternalEventService._calls
.find(c => c[0] === 'on' && c[1][0] === 'quantumCacheUpdated')
?.[1][1];
expect(fakeInternalEventService._calls).toContainEqual(['off', ['quantumCacheUpdated', callback]]);
});
it('should store in memory cache', async () => {
const cache = makeCache<string>();
await cache.set('foo', 'bar');
await cache.set('alpha', 'omega');
const result1 = await cache.get('foo');
const result2 = await cache.get('alpha');
expect(result1).toBe('bar');
expect(result2).toBe('omega');
});
it('should emit event when storing', async () => {
const cache = makeCache<string>({ name: 'fake' });
await cache.set('foo', 'bar');
expect(fakeInternalEventService._calls).toContainEqual(['emit', ['quantumCacheUpdated', { name: 'fake', keys: ['foo'] }]]);
});
it('should call onChanged when storing', async () => {
const fakeOnChanged = jest.fn(() => Promise.resolve());
const cache = makeCache<string>({
name: 'fake',
onChanged: fakeOnChanged,
});
await cache.set('foo', 'bar');
expect(fakeOnChanged).toHaveBeenCalledWith(['foo'], cache);
});
it('should not emit event when storing unchanged value', async () => {
const cache = makeCache<string>({ name: 'fake' });
await cache.set('foo', 'bar');
await cache.set('foo', 'bar');
expect(fakeInternalEventService._calls.filter(c => c[0] === 'emit')).toHaveLength(1);
});
it('should not call onChanged when storing unchanged value', async () => {
const fakeOnChanged = jest.fn(() => Promise.resolve());
const cache = makeCache<string>({
name: 'fake',
onChanged: fakeOnChanged,
});
await cache.set('foo', 'bar');
await cache.set('foo', 'bar');
expect(fakeOnChanged).toHaveBeenCalledTimes(1);
});
it('should fetch an unknown value', async () => {
const cache = makeCache<string>({
name: 'fake',
fetcher: key => `value#${key}`,
});
const result = await cache.fetch('foo');
expect(result).toBe('value#foo');
});
it('should store fetched value in memory cache', async () => {
const cache = makeCache<string>({
name: 'fake',
fetcher: key => `value#${key}`,
});
await cache.fetch('foo');
const result = cache.has('foo');
expect(result).toBe(true);
});
it('should call onChanged when fetching', async () => {
const fakeOnChanged = jest.fn(() => Promise.resolve());
const cache = makeCache<string>({
name: 'fake',
fetcher: key => `value#${key}`,
onChanged: fakeOnChanged,
});
await cache.fetch('foo');
expect(fakeOnChanged).toHaveBeenCalledWith(['foo'], cache);
});
it('should not emit event when fetching', async () => {
const cache = makeCache<string>({
name: 'fake',
fetcher: key => `value#${key}`,
});
await cache.fetch('foo');
expect(fakeInternalEventService._calls).not.toContainEqual(['emit', ['quantumCacheUpdated', { name: 'fake', keys: ['foo'] }]]);
});
it('should delete from memory cache', async () => {
const cache = makeCache<string>();
await cache.set('foo', 'bar');
await cache.delete('foo');
const result = cache.has('foo');
expect(result).toBe(false);
});
it('should call onChanged when deleting', async () => {
const fakeOnChanged = jest.fn(() => Promise.resolve());
const cache = makeCache<string>({
name: 'fake',
onChanged: fakeOnChanged,
});
await cache.set('foo', 'bar');
await cache.delete('foo');
expect(fakeOnChanged).toHaveBeenCalledWith(['foo'], cache);
});
it('should emit event when deleting', async () => {
const cache = makeCache<string>({ name: 'fake' });
await cache.set('foo', 'bar');
await cache.delete('foo');
expect(fakeInternalEventService._calls).toContainEqual(['emit', ['quantumCacheUpdated', { name: 'fake', keys: ['foo'] }]]);
});
it('should delete when receiving set event', async () => {
const cache = makeCache<string>({ name: 'fake' });
await cache.set('foo', 'bar');
await fakeInternalEventService._emitRedis('quantumCacheUpdated', { name: 'fake', keys: ['foo'] });
const result = cache.has('foo');
expect(result).toBe(false);
});
it('should call onChanged when receiving set event', async () => {
const fakeOnChanged = jest.fn(() => Promise.resolve());
const cache = makeCache<string>({
name: 'fake',
onChanged: fakeOnChanged,
});
await fakeInternalEventService._emitRedis('quantumCacheUpdated', { name: 'fake', keys: ['foo'] });
expect(fakeOnChanged).toHaveBeenCalledWith(['foo'], cache);
});
it('should delete when receiving delete event', async () => {
const cache = makeCache<string>({ name: 'fake' });
await cache.set('foo', 'bar');
await fakeInternalEventService._emitRedis('quantumCacheUpdated', { name: 'fake', keys: ['foo'] });
const result = cache.has('foo');
expect(result).toBe(false);
});
it('should call onChanged when receiving delete event', async () => {
const fakeOnChanged = jest.fn(() => Promise.resolve());
const cache = makeCache<string>({
name: 'fake',
onChanged: fakeOnChanged,
});
await cache.set('foo', 'bar');
await fakeInternalEventService._emitRedis('quantumCacheUpdated', { name: 'fake', keys: ['foo'] });
expect(fakeOnChanged).toHaveBeenCalledWith(['foo'], cache);
});
describe('get', () => {
it('should return value if present', async () => {
const cache = makeCache<string>();
await cache.set('foo', 'bar');
const result = cache.get('foo');
expect(result).toBe('bar');
});
it('should return undefined if missing', () => {
const cache = makeCache<string>();
const result = cache.get('foo');
expect(result).toBe(undefined);
});
});
describe('setMany', () => {
it('should populate all values', async () => {
const cache = makeCache<string>();
await cache.setMany([['foo', 'bar'], ['alpha', 'omega']]);
expect(cache.has('foo')).toBe(true);
expect(cache.has('alpha')).toBe(true);
});
it('should emit one event', async () => {
const cache = makeCache<string>({
name: 'fake',
});
await cache.setMany([['foo', 'bar'], ['alpha', 'omega']]);
expect(fakeInternalEventService._calls).toContainEqual(['emit', ['quantumCacheUpdated', { name: 'fake', keys: ['foo', 'alpha'] }]]);
expect(fakeInternalEventService._calls.filter(c => c[0] === 'emit')).toHaveLength(1);
});
it('should call onChanged once with all items', async () => {
const fakeOnChanged = jest.fn(() => Promise.resolve());
const cache = makeCache<string>({
name: 'fake',
onChanged: fakeOnChanged,
});
await cache.setMany([['foo', 'bar'], ['alpha', 'omega']]);
expect(fakeOnChanged).toHaveBeenCalledWith(['foo', 'alpha'], cache);
expect(fakeOnChanged).toHaveBeenCalledTimes(1);
});
it('should emit events only for changed items', async () => {
const fakeOnChanged = jest.fn(() => Promise.resolve());
const cache = makeCache<string>({
name: 'fake',
onChanged: fakeOnChanged,
});
await cache.set('foo', 'bar');
fakeOnChanged.mockClear();
fakeInternalEventService._reset();
await cache.setMany([['foo', 'bar'], ['alpha', 'omega']]);
expect(fakeInternalEventService._calls).toContainEqual(['emit', ['quantumCacheUpdated', { name: 'fake', keys: ['alpha'] }]]);
expect(fakeInternalEventService._calls.filter(c => c[0] === 'emit')).toHaveLength(1);
expect(fakeOnChanged).toHaveBeenCalledWith(['alpha'], cache);
expect(fakeOnChanged).toHaveBeenCalledTimes(1);
});
});
describe('getMany', () => {
it('should return empty for empty input', () => {
const cache = makeCache();
const result = cache.getMany([]);
expect(result).toEqual([]);
});
it('should return the value for all keys', () => {
const cache = makeCache();
cache.add('foo', 'bar');
cache.add('alpha', 'omega');
const result = cache.getMany(['foo', 'alpha']);
expect(result).toEqual([['foo', 'bar'], ['alpha', 'omega']]);
});
it('should return undefined for missing keys', () => {
const cache = makeCache();
cache.add('foo', 'bar');
const result = cache.getMany(['foo', 'alpha']);
expect(result).toEqual([['foo', 'bar'], ['alpha', undefined]]);
});
});
describe('fetchMany', () => {
it('should do nothing for empty input', async () => {
const fakeOnChanged = jest.fn(() => Promise.resolve());
const cache = makeCache({
onChanged: fakeOnChanged,
});
await cache.fetchMany([]);
expect(fakeOnChanged).not.toHaveBeenCalled();
expect(fakeInternalEventService._calls.filter(c => c[0] === 'emit')).toHaveLength(0);
});
it('should return existing items', async () => {
const cache = makeCache();
cache.add('foo', 'bar');
cache.add('alpha', 'omega');
const result = await cache.fetchMany(['foo', 'alpha']);
expect(result).toEqual([['foo', 'bar'], ['alpha', 'omega']]);
});
it('should return existing items without events', async () => {
const fakeOnChanged = jest.fn(() => Promise.resolve());
const cache = makeCache({
onChanged: fakeOnChanged,
});
cache.add('foo', 'bar');
cache.add('alpha', 'omega');
await cache.fetchMany(['foo', 'alpha']);
expect(fakeOnChanged).not.toHaveBeenCalled();
expect(fakeInternalEventService._calls.filter(c => c[0] === 'emit')).toHaveLength(0);
});
it('should call bulkFetcher for missing items', async () => {
const cache = makeCache({
bulkFetcher: keys => keys.map(k => [k, `${k}#many`]),
fetcher: key => `${key}#single`,
});
const results = await cache.fetchMany(['foo', 'alpha']);
expect(results).toEqual([['foo', 'foo#many'], ['alpha', 'alpha#many']]);
});
it('should call bulkFetcher only once', async () => {
const mockBulkFetcher = jest.fn((keys: string[]) => keys.map(k => [k, `${k}#value`] as [string, string]));
const cache = makeCache({
bulkFetcher: mockBulkFetcher,
});
await cache.fetchMany(['foo', 'bar']);
expect(mockBulkFetcher).toHaveBeenCalledTimes(1);
});
it('should call fetcher when fetchMany is undefined', async () => {
const cache = makeCache({
fetcher: key => `${key}#single`,
});
const results = await cache.fetchMany(['foo', 'alpha']);
expect(results).toEqual([['foo', 'foo#single'], ['alpha', 'alpha#single']]);
});
it('should call onChanged', async () => {
const fakeOnChanged = jest.fn(() => Promise.resolve());
const cache = makeCache({
onChanged: fakeOnChanged,
fetcher: k => k,
});
await cache.fetchMany(['foo', 'alpha']);
expect(fakeOnChanged).toHaveBeenCalledWith(['foo', 'alpha'], cache);
expect(fakeOnChanged).toHaveBeenCalledTimes(1);
});
it('should call onChanged only for changed', async () => {
const fakeOnChanged = jest.fn(() => Promise.resolve());
const cache = makeCache({
onChanged: fakeOnChanged,
fetcher: k => k,
});
cache.add('foo', 'bar');
await cache.fetchMany(['foo', 'alpha']);
expect(fakeOnChanged).toHaveBeenCalledWith(['alpha'], cache);
expect(fakeOnChanged).toHaveBeenCalledTimes(1);
});
it('should not emit event', async () => {
const cache = makeCache({
fetcher: k => k,
});
await cache.fetchMany(['foo', 'alpha']);
expect(fakeInternalEventService._calls.filter(c => c[0] === 'emit')).toHaveLength(0);
});
});
describe('refreshMany', () => {
it('should do nothing for empty input', async () => {
const fakeOnChanged = jest.fn(() => Promise.resolve());
const cache = makeCache({
onChanged: fakeOnChanged,
});
const result = await cache.refreshMany([]);
expect(result).toEqual([]);
expect(fakeOnChanged).not.toHaveBeenCalled();
expect(fakeInternalEventService._calls.filter(c => c[0] === 'emit')).toHaveLength(0);
});
it('should call bulkFetcher for all keys', async () => {
const mockBulkFetcher = jest.fn((keys: string[]) => keys.map(k => [k, `${k}#value`] as [string, string]));
const cache = makeCache({
bulkFetcher: mockBulkFetcher,
});
const result = await cache.refreshMany(['foo', 'alpha']);
expect(result).toEqual([['foo', 'foo#value'], ['alpha', 'alpha#value']]);
expect(mockBulkFetcher).toHaveBeenCalledWith(['foo', 'alpha'], cache);
expect(mockBulkFetcher).toHaveBeenCalledTimes(1);
});
it('should replace any existing keys', async () => {
const mockBulkFetcher = jest.fn((keys: string[]) => keys.map(k => [k, `${k}#value`] as [string, string]));
const cache = makeCache({
bulkFetcher: mockBulkFetcher,
});
cache.add('foo', 'bar');
const result = await cache.refreshMany(['foo', 'alpha']);
expect(result).toEqual([['foo', 'foo#value'], ['alpha', 'alpha#value']]);
expect(mockBulkFetcher).toHaveBeenCalledWith(['foo', 'alpha'], cache);
expect(mockBulkFetcher).toHaveBeenCalledTimes(1);
});
it('should call onChanged for all keys', async () => {
const fakeOnChanged = jest.fn(() => Promise.resolve());
const cache = makeCache({
bulkFetcher: keys => keys.map(k => [k, `${k}#value`]),
onChanged: fakeOnChanged,
});
cache.add('foo', 'bar');
await cache.refreshMany(['foo', 'alpha']);
expect(fakeOnChanged).toHaveBeenCalledWith(['foo', 'alpha'], cache);
expect(fakeOnChanged).toHaveBeenCalledTimes(1);
});
it('should emit event for all keys', async () => {
const cache = makeCache({
name: 'fake',
bulkFetcher: keys => keys.map(k => [k, `${k}#value`]),
});
cache.add('foo', 'bar');
await cache.refreshMany(['foo', 'alpha']);
expect(fakeInternalEventService._calls).toContainEqual(['emit', ['quantumCacheUpdated', { name: 'fake', keys: ['foo', 'alpha'] }]]);
expect(fakeInternalEventService._calls.filter(c => c[0] === 'emit')).toHaveLength(1);
});
});
describe('deleteMany', () => {
it('should remove keys from memory cache', async () => {
const cache = makeCache<string>();
await cache.set('foo', 'bar');
await cache.set('alpha', 'omega');
await cache.deleteMany(['foo', 'alpha']);
expect(cache.has('foo')).toBe(false);
expect(cache.has('alpha')).toBe(false);
});
it('should emit only one event', async () => {
const cache = makeCache<string>({
name: 'fake',
});
await cache.deleteMany(['foo', 'alpha']);
expect(fakeInternalEventService._calls).toContainEqual(['emit', ['quantumCacheUpdated', { name: 'fake', keys: ['foo', 'alpha'] }]]);
expect(fakeInternalEventService._calls.filter(c => c[0] === 'emit')).toHaveLength(1);
});
it('should call onChanged once with all items', async () => {
const fakeOnChanged = jest.fn(() => Promise.resolve());
const cache = makeCache<string>({
name: 'fake',
onChanged: fakeOnChanged,
});
await cache.deleteMany(['foo', 'alpha']);
expect(fakeOnChanged).toHaveBeenCalledWith(['foo', 'alpha'], cache);
expect(fakeOnChanged).toHaveBeenCalledTimes(1);
});
it('should do nothing if no keys are provided', async () => {
const fakeOnChanged = jest.fn(() => Promise.resolve());
const cache = makeCache<string>({
name: 'fake',
onChanged: fakeOnChanged,
});
await cache.deleteMany([]);
expect(fakeOnChanged).not.toHaveBeenCalled();
expect(fakeInternalEventService._calls.filter(c => c[0] === 'emit')).toHaveLength(0);
});
});
describe('refresh', () => {
it('should populate the value', async () => {
const cache = makeCache<string>({
name: 'fake',
fetcher: key => `value#${key}`,
});
await cache.refresh('foo');
const result = cache.has('foo');
expect(result).toBe(true);
});
it('should return the value', async () => {
const cache = makeCache<string>({
name: 'fake',
fetcher: key => `value#${key}`,
});
const result = await cache.refresh('foo');
expect(result).toBe('value#foo');
});
it('should replace the value if it exists', async () => {
const cache = makeCache<string>({
name: 'fake',
fetcher: key => `value#${key}`,
});
await cache.set('foo', 'bar');
const result = await cache.refresh('foo');
expect(result).toBe('value#foo');
});
it('should call onChanged', async () => {
const fakeOnChanged = jest.fn(() => Promise.resolve());
const cache = makeCache<string>({
name: 'fake',
fetcher: key => `value#${key}`,
onChanged: fakeOnChanged,
});
await cache.refresh('foo');
expect(fakeOnChanged).toHaveBeenCalledWith(['foo'], cache);
});
it('should emit event', async () => {
const cache = makeCache<string>({
name: 'fake',
fetcher: key => `value#${key}`,
});
await cache.refresh('foo');
expect(fakeInternalEventService._calls).toContainEqual(['emit', ['quantumCacheUpdated', { name: 'fake', keys: ['foo'] }]]);
});
});
describe('add', () => {
it('should add the item', () => {
const cache = makeCache();
cache.add('foo', 'bar');
expect(cache.has('foo')).toBe(true);
});
it('should not emit event', () => {
const cache = makeCache({
name: 'fake',
});
cache.add('foo', 'bar');
expect(fakeInternalEventService._calls.filter(c => c[0] === 'emit')).toHaveLength(0);
});
it('should not call onChanged', () => {
const fakeOnChanged = jest.fn(() => Promise.resolve());
const cache = makeCache({
onChanged: fakeOnChanged,
});
cache.add('foo', 'bar');
expect(fakeOnChanged).not.toHaveBeenCalled();
});
});
describe('addMany', () => {
it('should add all items', () => {
const cache = makeCache();
cache.addMany([['foo', 'bar'], ['alpha', 'omega']]);
expect(cache.has('foo')).toBe(true);
expect(cache.has('alpha')).toBe(true);
});
it('should not emit event', () => {
const cache = makeCache({
name: 'fake',
});
cache.addMany([['foo', 'bar'], ['alpha', 'omega']]);
expect(fakeInternalEventService._calls.filter(c => c[0] === 'emit')).toHaveLength(0);
});
it('should not call onChanged', () => {
const fakeOnChanged = jest.fn(() => Promise.resolve());
const cache = makeCache({
onChanged: fakeOnChanged,
});
cache.addMany([['foo', 'bar'], ['alpha', 'omega']]);
expect(fakeOnChanged).not.toHaveBeenCalled();
});
});
describe('has', () => {
it('should return false when empty', () => {
const cache = makeCache();
const result = cache.has('foo');
expect(result).toBe(false);
});
it('should return false when value is not in memory', async () => {
const cache = makeCache<string>();
await cache.set('foo', 'bar');
const result = cache.has('alpha');
expect(result).toBe(false);
});
it('should return true when value is in memory', async () => {
const cache = makeCache<string>();
await cache.set('foo', 'bar');
const result = cache.has('foo');
expect(result).toBe(true);
});
});
describe('size', () => {
it('should return 0 when empty', () => {
const cache = makeCache();
expect(cache.size).toBe(0);
});
it('should return correct size when populated', async () => {
const cache = makeCache<string>();
await cache.set('foo', 'bar');
expect(cache.size).toBe(1);
});
});
describe('entries', () => {
it('should return empty when empty', () => {
const cache = makeCache();
const result = Array.from(cache.entries());
expect(result).toHaveLength(0);
});
it('should return all entries when populated', async () => {
const cache = makeCache<string>();
await cache.set('foo', 'bar');
const result = Array.from(cache.entries());
expect(result).toEqual([['foo', 'bar']]);
});
});
describe('keys', () => {
it('should return empty when empty', () => {
const cache = makeCache();
const result = Array.from(cache.keys());
expect(result).toHaveLength(0);
});
it('should return all keys when populated', async () => {
const cache = makeCache<string>();
await cache.set('foo', 'bar');
const result = Array.from(cache.keys());
expect(result).toEqual(['foo']);
});
});
describe('values', () => {
it('should return empty when empty', () => {
const cache = makeCache();
const result = Array.from(cache.values());
expect(result).toHaveLength(0);
});
it('should return all values when populated', async () => {
const cache = makeCache<string>();
await cache.set('foo', 'bar');
const result = Array.from(cache.values());
expect(result).toEqual(['bar']);
});
});
describe('[Symbol.iterator]', () => {
it('should return empty when empty', () => {
const cache = makeCache();
const result = Array.from(cache);
expect(result).toHaveLength(0);
});
it('should return all entries when populated', async () => {
const cache = makeCache<string>();
await cache.set('foo', 'bar');
const result = Array.from(cache);
expect(result).toEqual([['foo', 'bar']]);
});
});
});