fix(review): 处理全量审查剩余项（M4-M8/M11/M12 + 21 条 low）
后端
- asset_service: delete_asset 先清 subdomains/site_certs 子表（SQLite 不强制
FK 且 assets.id 无 AUTOINCREMENT，删最大 id 后新建会复用 rowid 导致孤儿误挂）；
metrics.db 清理改 best-effort，不再让已成功的删除返回 500
- provider_service: slug 唯一约束竞态补 IntegrityError 兜底（与 account_service 一致）
- account_service/sync_service: raise HTTPException 补 from e，保留原始异常链
- sync_service: 删死代码 _load_config；新增 BalanceUnsupported，平台无余额接口时
计入 skipped 而不是静默当作失败；到期日写库前归一
- ai: 删掉 OpenAI 早已下线的 billing 余额接口实现，能力探测如实报「不支持」
- adapters/base: _overridden 改走 MRO，中间层基类实现的能力不再被误判为未实现
- core/timeutils: 删死代码 utcnow_iso（注释声称 Z 后缀，实际输出 +00:00）
- main: lifespan 退出时 await 被 cancel 的清理任务，避免 pending task 被销毁
- ssl_service: 写 site_certs/subdomains 前校验 asset_id 存在，杜绝悬空引用；
到期阈值贯穿 _judge_status/_apply_probe/check_one/check_all_site_certs
- routers/credentials、accounts: 明文密码 GET 补 Cache-Control: no-store
- routers/stats: /stats/expiring 的 Query 描述与实际语义对齐（不设下界，含已过期）
Agent
- collector: 解析 sshd_config 的 Include（含 sshd_config.d/*.conf，按 sshd 语义
「首次出现的值生效」），未显式配置的关键字按编译默认值判定，消除假 pass；
防火墙检测改用 iptables -S 统计 -A 规则行并读 INPUT 策略（-L -n 恒有链头输出）
- reporter: 4xx（除 429）不再重试直接退出，429 优先按 Retry-After 退避
安全与部署
- core/security: require_agent_key 改为对外 fail-closed——tailnet/本机放行，
外部来源未配 AGENT_KEY 时一律 401（此前全放行，而服务默认监听 0.0.0.0，
公网上任何人可伪造监控指标与安全检查结果）
- main: 启动时把「未配 API_KEY/AGENT_KEY 则外部来源不可用」打进日志
- run.sh: HOST/PORT 可用环境变量覆盖，并说明默认 0.0.0.0 的权衡
- deploy/setup.sh: 新装时自动生成随机 API_KEY/AGENT_KEY 并提示同步到各 VPS；
注册 vps-cert-check.timer
- deploy/vps-cert-check.{service,timer}: 新增每日 09:30 的站点证书到期探测
（此前 check_site_certs.py 没有任何 unit，证书提醒不会自动跑）
- deploy/update.sh: sync_units 对 timer 用 enable --now（新增 timer 只 restart
不会建开机自启软链），主服务 unit 变更后额外 restart；pip 安装失败不再被吞
- scripts/check_site_certs.py: threshold 参数真正生效并打印
- scripts/make_migration_bundle.py: 缺 assets.db 直接 abort；umask 077 + chmod 600
（包内含 .env 的 MASTER_KEY 与 escrow，不能默认 644 落在 /tmp）
- scripts/restore_migration_bundle.py: 校验之外补必备文件预检，缺文件在触碰
目标机状态之前就 abort
前端
- store: 拆分列表口径 assets 与统计口径 allAssets（无筛选时同源，不重复请求）；
删除已无消费者的 expiring/loadExpiring（每次 loadAll 白发一个请求）
- dashboard/monitor/providers/subscriptions/modals: 统计与计数改用 allAssets，
资产页的筛选（含统计卡片点击跳转带的筛选）不再让其他视图的数字缩水
- modals: 账号同步后补 loadOverview；settings: 导入后改 loadAll
- api: countdownText 支持 unit/empty 选项，monitor/domains 各自的实现改为委托，
assets/domains/subscriptions 的到期列不再直出「(-3天)」
- servers: cpu/mem/disk 百分比统一判空，采样缺项不再渲染「null%」
- assets: 同步 AI 余额的结果提示区分 skipped（平台无余额接口）与 failed
- modals: 修正凭据库头注释——2FA 动态码是选填 + 可事后补验，与实现一致
测试
- 新增 tests/test_agent_collector.py（8 条）、tests/test_security.py（11 条）
- 全量 113 passed；改动 JS 均通过 node --check，deploy 脚本通过 bash -n
- 浏览器实测：筛选态下总览/监控中心/平台页/付费资产的统计仍为全量口径，
到期文案与 null 百分比渲染正确，console 零报错
This commit is contained in:
gouki
@@ -113,13 +113,13 @@ def create_account(session: Session, data: AccountCreate) -> AccountRead:
|
||||
try:
|
||||
session.add(account)
|
||||
session.commit()
|
||||
except IntegrityError:
|
||||
except IntegrityError as e:
|
||||
# 并发创建同名账号时唯一约束兜底：检查与提交之间存在竞态窗口
|
||||
session.rollback()
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
detail=f"该平台下账号已存在：{name}",
|
||||
)
|
||||
) from e
|
||||
session.refresh(account)
|
||||
return _to_read(account, _asset_counts(session), _cred_of(session, account))
|
||||
|
||||
@@ -150,13 +150,13 @@ def update_account(session: Session, account_id: int, data: AccountUpdate) -> Ac
|
||||
try:
|
||||
session.add(account)
|
||||
session.commit()
|
||||
except IntegrityError:
|
||||
except IntegrityError as e:
|
||||
# 重命名撞上已有账号时唯一约束兜底，与 _check_name_taken 存在竞态窗口
|
||||
session.rollback()
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
detail=f"该平台下账号已存在：{account.name}",
|
||||
)
|
||||
) from e
|
||||
session.refresh(account)
|
||||
return _to_read(account, _asset_counts(session), _cred_of(session, account))
|
||||
|
||||
|
||||
@@ -299,17 +299,42 @@ def update_asset(session: Session, asset_id: int, data: AssetUpdate) -> AssetRea
|
||||
|
||||
|
||||
def delete_asset(session: Session, asset_id: int) -> None:
|
||||
"""删除资产及其详情，并清理 metrics.db 中的关联监控数据（避免孤儿数据）"""
|
||||
"""删除资产及其详情/子表，并尽力清理 metrics.db 中的关联监控数据（避免孤儿数据）"""
|
||||
asset = session.get(Asset, asset_id)
|
||||
if not asset:
|
||||
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="资产不存在")
|
||||
detail = _get_detail(session, asset)
|
||||
if detail is not None:
|
||||
session.delete(detail)
|
||||
_delete_child_rows(session, asset_id)
|
||||
session.delete(asset)
|
||||
session.commit()
|
||||
logger.info("删除资产 id=%s name=%s", asset_id, asset.name)
|
||||
_cleanup_metrics_for_asset(asset_id)
|
||||
# metrics.db 是独立库（可能正在被 Agent 写入而短暂锁定），
|
||||
# 清理失败不应让已经成功的删除返回 500，否则前端会误报「删除失败」
|
||||
try:
|
||||
_cleanup_metrics_for_asset(asset_id)
|
||||
except Exception as e: # noqa: BLE001
|
||||
logger.warning(
|
||||
"清理资产监控数据失败 asset_id=%s：%s（资产已删除，监控数据可后续手动清理）",
|
||||
asset_id,
|
||||
e,
|
||||
)
|
||||
|
||||
|
||||
def _delete_child_rows(session: Session, asset_id: int) -> None:
|
||||
"""删除 assets.db 中引用该资产的子表行（subdomains / site_certs）
|
||||
|
||||
必须显式清理：SQLite 默认不强制外键（PRAGMA foreign_keys=0），残留行会变成孤儿；
|
||||
而 assets.id 是 INTEGER PRIMARY KEY（无 AUTOINCREMENT），删掉最大 id 后
|
||||
新建资产必然复用该 id，孤儿行会被误挂到不相干的新资产上。
|
||||
"""
|
||||
from sqlmodel import delete
|
||||
|
||||
from app.models.ssl import SiteCert, Subdomain
|
||||
|
||||
for model in (Subdomain, SiteCert):
|
||||
session.exec(delete(model).where(model.asset_id == asset_id))
|
||||
|
||||
|
||||
def _cleanup_metrics_for_asset(asset_id: int) -> None:
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
from typing import List, Optional
|
||||
|
||||
from fastapi import HTTPException, status
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
from sqlmodel import Session, select
|
||||
|
||||
from app.core import crypto
|
||||
@@ -47,8 +48,16 @@ def create_provider(session: Session, data: ProviderCreate) -> ProviderRead:
|
||||
provider_data = data.model_dump(exclude={"api_config"})
|
||||
provider = Provider(**provider_data)
|
||||
provider.api_config_encrypted = crypto.encrypt(data.api_config)
|
||||
session.add(provider)
|
||||
session.commit()
|
||||
try:
|
||||
session.add(provider)
|
||||
session.commit()
|
||||
except IntegrityError as e:
|
||||
# 上面的 exists 检查与 commit 之间存在竞态窗口，靠 slug 唯一约束兜底，
|
||||
# 否则并发建同名平台会直接把 IntegrityError 抛成 500
|
||||
session.rollback()
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST, detail=f"slug 已存在：{data.slug}"
|
||||
) from e
|
||||
session.refresh(provider)
|
||||
return _to_read(provider)
|
||||
|
||||
@@ -64,8 +73,15 @@ def update_provider(
|
||||
setattr(provider, key, value)
|
||||
if data.api_config is not None:
|
||||
provider.api_config_encrypted = crypto.encrypt(data.api_config)
|
||||
session.add(provider)
|
||||
session.commit()
|
||||
try:
|
||||
session.add(provider)
|
||||
session.commit()
|
||||
except IntegrityError as e:
|
||||
# 改 slug 撞上已有平台时同样靠唯一约束兜底
|
||||
session.rollback()
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST, detail=f"slug 已存在：{provider.slug}"
|
||||
) from e
|
||||
session.refresh(provider)
|
||||
return _to_read(provider)
|
||||
|
||||
|
||||
@@ -29,13 +29,13 @@ def days_until(d: Optional[date]) -> Optional[int]:
|
||||
return (d - date.today()).days
|
||||
|
||||
|
||||
def _judge_status(days: Optional[int]) -> str:
|
||||
def _judge_status(days: Optional[int], threshold: int = EXPIRING_THRESHOLD) -> str:
|
||||
"""按剩余天数定级：expired / expiring / valid"""
|
||||
if days is None:
|
||||
return "unknown"
|
||||
if days < 0:
|
||||
return "expired"
|
||||
if days <= EXPIRING_THRESHOLD:
|
||||
if days <= threshold:
|
||||
return "expiring"
|
||||
return "valid"
|
||||
|
||||
@@ -115,7 +115,20 @@ def list_subdomains(session: Session, asset_id: Optional[int] = None) -> list:
|
||||
]
|
||||
|
||||
|
||||
def _require_asset(session: Session, asset_id: Optional[int]) -> None:
|
||||
"""校验关联资产存在
|
||||
|
||||
SQLite 默认不强制外键，写一个不存在的 asset_id 不会报错，
|
||||
但列表页 join 不出资产名，删除资产时也清不到这些悬空行。
|
||||
"""
|
||||
if asset_id is None:
|
||||
return
|
||||
if not session.get(Asset, asset_id):
|
||||
raise ValueError(f"关联资产不存在（id={asset_id}）")
|
||||
|
||||
|
||||
def create_subdomain(session: Session, data: dict) -> Subdomain:
|
||||
_require_asset(session, data["asset_id"])
|
||||
sub = Subdomain(
|
||||
asset_id=data["asset_id"],
|
||||
host=data["host"].strip().lower(),
|
||||
@@ -157,7 +170,7 @@ def delete_subdomain(session: Session, sub_id: int) -> None:
|
||||
|
||||
# ---------------- 站点证书监控 ----------------
|
||||
|
||||
def _apply_probe(cert: SiteCert, info: dict) -> SiteCert:
|
||||
def _apply_probe(cert: SiteCert, info: dict, threshold: int = EXPIRING_THRESHOLD) -> SiteCert:
|
||||
"""把探测结果写入模型并定级"""
|
||||
cert.subject_cn = info["subject_cn"]
|
||||
cert.issuer = info["issuer"]
|
||||
@@ -165,16 +178,16 @@ def _apply_probe(cert: SiteCert, info: dict) -> SiteCert:
|
||||
cert.valid_to = info["valid_to"]
|
||||
cert.fingerprint = info["fingerprint"]
|
||||
cert.error = None
|
||||
cert.status = _judge_status(days_until(info["valid_to"]))
|
||||
cert.status = _judge_status(days_until(info["valid_to"]), threshold)
|
||||
cert.last_checked_at = utcnow()
|
||||
return cert
|
||||
|
||||
|
||||
def check_one(session: Session, cert: SiteCert) -> SiteCert:
|
||||
def check_one(session: Session, cert: SiteCert, threshold: int = EXPIRING_THRESHOLD) -> SiteCert:
|
||||
"""重新探测单条证书记录（失败则标记 error，保留旧到期信息）"""
|
||||
try:
|
||||
info = probe_site_cert(cert.hostname, cert.port)
|
||||
_apply_probe(cert, info)
|
||||
_apply_probe(cert, info, threshold)
|
||||
except Exception as e: # noqa: BLE001
|
||||
cert.status = "error"
|
||||
cert.error = str(e)[:200]
|
||||
@@ -187,6 +200,7 @@ def check_one(session: Session, cert: SiteCert) -> SiteCert:
|
||||
|
||||
def create_site_cert(session: Session, hostname: str, port: int = 443, asset_id: Optional[int] = None) -> SiteCert:
|
||||
"""创建探测目标并立即探测一次；hostname+port 已存在则复用并刷新"""
|
||||
_require_asset(session, asset_id)
|
||||
hostname = hostname.strip().lower()
|
||||
existing = session.exec(
|
||||
select(SiteCert).where(SiteCert.hostname == hostname, SiteCert.port == port)
|
||||
@@ -223,12 +237,13 @@ def delete_site_cert(session: Session, cert_id: int) -> None:
|
||||
session.commit()
|
||||
|
||||
|
||||
def check_all_site_certs(session: Session) -> dict:
|
||||
def check_all_site_certs(session: Session, threshold: int = EXPIRING_THRESHOLD) -> dict:
|
||||
"""全量刷新所有站点证书，返回统计与异常清单
|
||||
|
||||
探测为纯网络 IO（单次最长 8s），用线程池并发探测后统一写库：
|
||||
串行时 N 个站点最坏耗时 N×8s，并发后接近单站点耗时；
|
||||
写库集中在主线程一次 commit（原来逐条 commit 产生 N 次事务）。
|
||||
threshold：「即将到期」的天数阈值，默认 EXPIRING_THRESHOLD。
|
||||
"""
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
|
||||
@@ -244,7 +259,7 @@ def check_all_site_certs(session: Session) -> dict:
|
||||
for future in futures:
|
||||
cert = futures[future]
|
||||
try:
|
||||
_apply_probe(cert, future.result())
|
||||
_apply_probe(cert, future.result(), threshold)
|
||||
except Exception as e: # noqa: BLE001
|
||||
cert.status = "error"
|
||||
cert.error = str(e)[:200]
|
||||
|
||||
@@ -36,17 +36,6 @@ from app.models.provider import Provider
|
||||
_VALID_STATUS = {s.value for s in AssetStatus}
|
||||
|
||||
|
||||
def _load_config(provider: Provider) -> dict:
|
||||
"""（已弃用）解密平台级 API 配置，仅为兼容历史数据保留"""
|
||||
plain = crypto.decrypt(provider.api_config_encrypted)
|
||||
if not plain:
|
||||
return {}
|
||||
try:
|
||||
return json.loads(plain)
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
return {}
|
||||
|
||||
|
||||
def _load_account_config(account: Account) -> dict:
|
||||
"""解密账号的 API 配置 JSON"""
|
||||
plain = crypto.decrypt(account.api_config_encrypted)
|
||||
@@ -334,6 +323,20 @@ _AI_SDK_MAP = {
|
||||
}
|
||||
|
||||
|
||||
class BalanceUnsupported(HTTPException):
|
||||
"""该平台没有可用的余额查询接口
|
||||
|
||||
与「查询失败」区分开：sync_all_ai_balances 把它计入 skipped 而不是 failed，
|
||||
否则每次「同步 AI 余额」都会把这类平台报成错误。
|
||||
"""
|
||||
|
||||
def __init__(self, sdk_type: str):
|
||||
super().__init__(
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
detail=f"{sdk_type} 暂不支持余额查询",
|
||||
)
|
||||
|
||||
|
||||
def refresh_ai_balance(session: Session, asset_id: int) -> dict:
|
||||
"""刷新 AI 账号余额：解密 api_key → 适配器 get_account → 更新余额"""
|
||||
asset = session.get(Asset, asset_id)
|
||||
@@ -367,19 +370,18 @@ def refresh_ai_balance(session: Session, asset_id: int) -> dict:
|
||||
)
|
||||
|
||||
adapter = registry.get_adapter(sdk_type, {"api_key": api_key})
|
||||
if not adapter.capabilities().get("get_account"):
|
||||
raise BalanceUnsupported(sdk_type)
|
||||
try:
|
||||
acc = adapter.get_account()
|
||||
except NotImplementedError:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
detail=f"{sdk_type} 暂不支持余额查询",
|
||||
)
|
||||
except NotImplementedError as e:
|
||||
raise BalanceUnsupported(sdk_type) from e
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as e: # noqa: BLE001
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_502_BAD_GATEWAY, detail=f"查询余额失败：{e}"
|
||||
)
|
||||
) from e
|
||||
|
||||
if acc.balance is not None:
|
||||
ai.balance = acc.balance
|
||||
@@ -417,6 +419,7 @@ def sync_all_ai_balances(session: Session, max_workers: int = 5) -> dict:
|
||||
|
||||
success = 0
|
||||
failed = 0
|
||||
skipped = 0
|
||||
errors = []
|
||||
with ThreadPoolExecutor(max_workers=max_workers) as pool:
|
||||
futures = {pool.submit(_refresh_one, aid): name for aid, name in asset_ids}
|
||||
@@ -425,6 +428,9 @@ def sync_all_ai_balances(session: Session, max_workers: int = 5) -> dict:
|
||||
try:
|
||||
future.result()
|
||||
success += 1
|
||||
except BalanceUnsupported:
|
||||
# 平台本身没余额接口，不算失败（否则每次都报一堆红）
|
||||
skipped += 1
|
||||
except HTTPException as e:
|
||||
failed += 1
|
||||
errors.append(f"{name}: {e.detail}")
|
||||
@@ -435,6 +441,7 @@ def sync_all_ai_balances(session: Session, max_workers: int = 5) -> dict:
|
||||
"total": len(asset_ids),
|
||||
"success": success,
|
||||
"failed": failed,
|
||||
"skipped": skipped,
|
||||
"errors": errors,
|
||||
"synced_at": utcnow().isoformat(),
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user