评估发现项目只有台账能力,缺失真正的多云管理链路。本次按缺口逐项补齐: 适配层 - 新增 aws / linode / namesilo 适配器,registry 登记;阿里云、腾讯云补 list_domains 与 get_account 余额 - NormalizedVPS 补 expiry_date + charge_type:此前云端同步进来的 VPS 拿不到 到期日,永远进不了续费提醒——而续费提醒正是这套台账存在的理由 - base.py 统一云端错误可读化(http_error_message / cloud_error_text):各厂商 把原因放的位置都不同(Linode errors[].reason、Cloudflare errors[].message、 腾讯 Response.Error.Message、AWS XML),此前直接 str(e) 会把 httpx 的多行文本 连同 MDN 链接弹给用户 动作层与 DNS - 新增 /api/actions:实例启停/重启/续费,写操作全部二次确认 + 审计 - 新增 Cloudflare DNS 记录的增删改查 定时同步 - scripts/sync_all_assets.py + /api/sync/assets + vps-sync-assets.timer(08:30) - 08:30 而非随机时刻:必须早于 09:00 的续费检查,否则提醒读到上一天的旧到期日 - sync_all_assets 修正计数:每一项子同步都失败时不再计为「成功」,否则凭证过期 会显示「成功 1」,用户以为数据拉到了、实际一条没有 数据一致性 - AssetRead 输出 external_id(只出不进):前端「云操作」「DNS」按钮靠它渲染, 缺了则所有按钮永远出不来 - import_assets 落库 external_id:导出带了去重键而导入丢弃,会让迁移后首次同步 把所有云端资产当新增再建一份 - _align_preset_providers 取代 _backfill_provider_services:库里声明但注册表未 实现的 sdk_type 清空(声明支持却不支持,比明说不支持更糟),services 只增量 补标签不覆盖,用户自建平台不动 前端如实展示 - cost 为 0 统一显示「未填」而非「0 USD」:0 既可能是没填、也可能是云端不返回 价格,而汇总处又把 0 当「无费用」排除,两处口径不一致 - 总览「本年支出」增加 year_cost_missing,把合计的覆盖缺口说出来 - DNS 弹窗补 MX/SRV 优先级与备注;列表请求失败时 provider 用资产自带 slug 兜底, 否则 Cloudflare 代理勾选框永远不出现 测试:257 -> 309 passed
205 lines
7.6 KiB
Python
205 lines
7.6 KiB
Python
"""资产导出 / 导入的往返一致性测试
|
||
|
||
export_assets 自称「用于数据迁移/备份」,那它导出的字段就必须在 import_assets
|
||
里真的落库。external_id 是 sync_service 的去重键 (provider_id, external_id, asset_type),
|
||
AssetCreate/AssetUpdate 不收这个字段——导入时若被 pydantic 静默丢弃,
|
||
迁移后首次同步会把所有云端资产当成新增再建一份,台账直接翻倍。
|
||
|
||
运行:.venv/bin/pytest tests/test_asset_export_import.py -v
|
||
"""
|
||
|
||
import pytest
|
||
from sqlmodel import Session, SQLModel, create_engine, select
|
||
|
||
import app.models.asset # noqa: F401
|
||
import app.models.credential # noqa: F401
|
||
import app.models.monitor # noqa: F401
|
||
import app.models.provider # noqa: F401
|
||
import app.models.ssl # noqa: F401
|
||
from app.models.asset import Asset, AssetType, DomainDetail, VPSDetail
|
||
from app.services.asset_service import export_assets, import_assets
|
||
|
||
|
||
@pytest.fixture()
|
||
def engine(tmp_path, monkeypatch):
|
||
eng = create_engine(
|
||
f"sqlite:///{tmp_path / 'assets.db'}", connect_args={"check_same_thread": False}
|
||
)
|
||
SQLModel.metadata.create_all(eng)
|
||
# _cleanup_metrics_for_asset 在函数内部取 metrics_engine,指到临时库避免污染本地 data/
|
||
monkeypatch.setattr("app.database.metrics_engine", eng)
|
||
yield eng
|
||
eng.dispose()
|
||
|
||
|
||
def _seed_vps(engine, name="web-01", external_id="i-abc123"):
|
||
with Session(engine) as s:
|
||
a = Asset(
|
||
name=name, asset_type=AssetType.VPS, provider="vultr",
|
||
external_id=external_id, cost=5.0, currency="USD",
|
||
)
|
||
s.add(a)
|
||
s.commit()
|
||
s.refresh(a)
|
||
s.add(VPSDetail(asset_id=a.id, ip_address="1.2.3.4"))
|
||
s.commit()
|
||
return a.id
|
||
|
||
|
||
def _seed_domain(engine, name="example.com", external_id="example.com"):
|
||
with Session(engine) as s:
|
||
a = Asset(name=name, asset_type=AssetType.DOMAIN, provider="namesilo", external_id=external_id)
|
||
s.add(a)
|
||
s.commit()
|
||
s.refresh(a)
|
||
s.add(DomainDetail(asset_id=a.id, domain_name=name))
|
||
s.commit()
|
||
return a.id
|
||
|
||
|
||
def _external_ids(engine):
|
||
with Session(engine) as s:
|
||
return {a.name: a.external_id for a in s.exec(select(Asset)).all()}
|
||
|
||
|
||
def _wipe(engine):
|
||
"""清空资产与详情行
|
||
|
||
必须连详情一起删:SQLite 默认不强制外键,只删 Asset 会留下孤儿详情行;
|
||
而 assets.id 是 INTEGER PRIMARY KEY(无 AUTOINCREMENT),删掉后新建资产
|
||
必然复用同一个 id,撞上孤儿行的 UNIQUE(asset_id)。
|
||
"""
|
||
with Session(engine) as s:
|
||
for model in (VPSDetail, DomainDetail):
|
||
for row in s.exec(select(model)).all():
|
||
s.delete(row)
|
||
for a in s.exec(select(Asset)).all():
|
||
s.delete(a)
|
||
s.commit()
|
||
|
||
|
||
# ----------------------------- 导出 ----------------------------- #
|
||
def test_export_includes_external_id(engine):
|
||
_seed_vps(engine)
|
||
with Session(engine) as s:
|
||
data = export_assets(s)
|
||
assert data["count"] == 1
|
||
assert data["assets"][0]["external_id"] == "i-abc123"
|
||
|
||
|
||
def test_export_carries_detail_and_count(engine):
|
||
_seed_vps(engine)
|
||
_seed_domain(engine)
|
||
with Session(engine) as s:
|
||
data = export_assets(s)
|
||
assert data["count"] == 2
|
||
by_name = {a["name"]: a for a in data["assets"]}
|
||
assert by_name["web-01"]["vps_detail"]["ip_address"] == "1.2.3.4"
|
||
assert by_name["example.com"]["domain_detail"]["domain_name"] == "example.com"
|
||
|
||
|
||
# ----------------------------- 导入落库 ----------------------------- #
|
||
def test_import_creates_asset_with_external_id(engine):
|
||
"""空库导入:external_id 必须落到行上,不能被 AssetCreate 静默吞掉"""
|
||
_seed_vps(engine)
|
||
with Session(engine) as s:
|
||
payload = export_assets(s)["assets"]
|
||
_wipe(engine)
|
||
with Session(engine) as s:
|
||
result = import_assets(s, payload)
|
||
assert result["errors"] == []
|
||
assert result["created"] == 1
|
||
assert _external_ids(engine)["web-01"] == "i-abc123"
|
||
|
||
|
||
def test_import_updates_existing_and_keeps_external_id(engine):
|
||
"""同名同类型同平台 -> 走更新分支,external_id 不能丢"""
|
||
asset_id = _seed_vps(engine)
|
||
with Session(engine) as s:
|
||
payload = export_assets(s)["assets"]
|
||
with Session(engine) as s:
|
||
a = s.get(Asset, asset_id)
|
||
a.external_id = None # 模拟历史数据没有 external_id
|
||
s.add(a)
|
||
s.commit()
|
||
with Session(engine) as s:
|
||
result = import_assets(s, payload)
|
||
assert result["updated"] == 1 and result["created"] == 0
|
||
assert _external_ids(engine)["web-01"] == "i-abc123"
|
||
|
||
|
||
def test_import_does_not_clobber_a_fresher_external_id(engine):
|
||
"""已有值时不覆盖:导入包可能比线上同步到的旧,覆盖会让去重键指向已销毁的实例"""
|
||
asset_id = _seed_vps(engine, external_id="i-old")
|
||
with Session(engine) as s:
|
||
payload = export_assets(s)["assets"]
|
||
with Session(engine) as s:
|
||
a = s.get(Asset, asset_id)
|
||
a.external_id = "i-new"
|
||
s.add(a)
|
||
s.commit()
|
||
with Session(engine) as s:
|
||
import_assets(s, payload)
|
||
assert _external_ids(engine)["web-01"] == "i-new"
|
||
|
||
|
||
def test_import_ignores_computed_fields(engine):
|
||
"""days_to_expiry / provider_name / account_name 是派生值,导入时必须跳过而不是写库"""
|
||
_seed_vps(engine)
|
||
with Session(engine) as s:
|
||
payload = export_assets(s)["assets"]
|
||
payload[0]["days_to_expiry"] = 999
|
||
payload[0]["provider_name"] = "不存在的平台"
|
||
payload[0]["account_name"] = "不存在的账号"
|
||
_wipe(engine)
|
||
with Session(engine) as s:
|
||
result = import_assets(s, payload)
|
||
assert result["errors"] == []
|
||
assert result["created"] == 1
|
||
|
||
|
||
def test_import_without_external_id_key_still_works(engine):
|
||
"""老导出包没有这个字段,不能因此报错"""
|
||
payload = [{
|
||
"name": "legacy", "asset_type": "vps", "provider": "vultr",
|
||
"vps_detail": {"ip_address": "9.9.9.9"},
|
||
}]
|
||
with Session(engine) as s:
|
||
result = import_assets(s, payload)
|
||
assert result["created"] == 1 and result["errors"] == []
|
||
assert _external_ids(engine)["legacy"] is None
|
||
|
||
|
||
def test_import_all_or_nothing_on_failure(engine):
|
||
"""单事务语义:一条失败则整体回滚,不能留下半截数据"""
|
||
_seed_vps(engine, name="good")
|
||
payload = [
|
||
{"name": "ok-1", "asset_type": "vps", "provider": "vultr",
|
||
"external_id": "i-1", "vps_detail": {"ip_address": "1.1.1.1"}},
|
||
{"name": "bad-1", "asset_type": "vps", "provider": "vultr"}, # 缺 vps_detail -> 400
|
||
]
|
||
with Session(engine) as s:
|
||
result = import_assets(s, payload)
|
||
assert result["created"] == 0 and result["updated"] == 0
|
||
assert len(result["errors"]) == 1
|
||
names = _external_ids(engine)
|
||
assert "ok-1" not in names # 成功那条也回滚了
|
||
assert set(names) == {"good"}
|
||
|
||
|
||
def test_round_trip_preserves_everything(engine):
|
||
"""完整往返:导出 -> 清空 -> 导入,external_id 与详情都在"""
|
||
_seed_vps(engine, name="web-01", external_id="i-111")
|
||
_seed_domain(engine, name="example.com", external_id="example.com")
|
||
with Session(engine) as s:
|
||
payload = export_assets(s)["assets"]
|
||
_wipe(engine)
|
||
with Session(engine) as s:
|
||
result = import_assets(s, payload)
|
||
assert result["errors"] == [] and result["created"] == 2
|
||
assert _external_ids(engine) == {"web-01": "i-111", "example.com": "example.com"}
|
||
with Session(engine) as s:
|
||
vps = s.exec(select(Asset).where(Asset.name == "web-01")).one()
|
||
detail = s.exec(select(VPSDetail).where(VPSDetail.asset_id == vps.id)).one()
|
||
assert detail.ip_address == "1.2.3.4"
|