> Исторический документ исходного WTG. Сервис удалён 07.09.2026; описанные здесь команды эксплуатации не являются действующей конфигурацией VPS.

# Code Map

Актуально на 2026-06-22. Этот файл фиксирует текущую реализацию WTG, чтобы
новые изменения сверялись с реальным кодом, а не со старыми `/review/*` или
stage 2/stage 3 документами.

## CLI apps and service roles

- `wtg-web` -> `src/wtg/cli/web.py`: FastAPI/Jinja admin.
- `wtg-admin-worker` -> `src/wtg/cli/admin_worker.py`: claims `admin_jobs`,
  запускает задачи и поддерживает heartbeat/stale cleanup.
- `wtg-scheduler` -> `src/wtg/cli/scheduler.py`: production API polling
  scheduler. Ставит validation/token refresh/API sync jobs.
- `wtg-auth` -> `src/wtg/cli/auth.py`: manual VK login and session validation.
- `wtg-crawler` -> `src/wtg/cli/crawler.py`: manual/debug crawler commands.
- `wtg-report` -> `src/wtg/cli/report.py`: manual-test export.
- `wtg-maintenance` -> `src/wtg/cli/maintenance.py`: reset, backlog,
  API pool sync, stale archive, publication teacher and dataset commands.
- `wtg-ocr-sidecar` -> `src/wtg/cli/ocr_sidecar.py`: PaddleOCR FastAPI sidecar
  with `/health` and `/ocr`.
- `wtg-ocr-worker` -> `src/wtg/cli/ocr_worker.py`: leases `ocr_jobs` and calls
  sidecar.
- `wtg-fast-screen-pool` -> `src/wtg/cli/fast_screen_pool.py`: global pool
  fast wall checks.
- `wtg-accounts-vnc` -> `src/wtg/cli/accounts_vnc.py`: legacy visible account
  lanes, production profile `legacy-vnc`.

## Settings

Main config lives in `src/wtg/settings.py`.

Important groups:

- state/deps/auth paths: `WTG_STATE_ROOT`, `WTG_DEPS_ROOT`,
  `WTG_STORAGE_STATE_PATH`, `WTG_AUTH_COOKIES_PATH`,
  `WTG_AUTH_MANIFEST_PATH`;
- media/reports/logs/archive: `WTG_POST_MEDIA_ROOT`, `WTG_REPORTS_PATH`,
  `WTG_RUN_LOGS_PATH`, `WTG_ARCHIVE_ROOT`;
- VK/API: `VK_BASE_URL`, `VK_GROUPS_CATALOG_URL`, `VK_API_BASE_URL`,
  `VK_API_VERSION`, `VK_SERVICE_TOKEN`, `WTG_VK_SERVICE_TOKEN`;
- scheduler/API: `WTG_API_POSTS_POLL_ENABLED`,
  `WTG_API_USER_TOKEN_REFRESH_ENABLED`, `WTG_API_POSTS_*`;
- OCR: `WTG_OCR_*`, `WTG_PADDLE_OCR_*`, `WTG_OCR_SIDECAR_URL`;
- fast screen pool: `WTG_FAST_SCREEN_POOL_*`;
- LLM/OpenRouter: `WTG_LLM_*`, `OPENROUTER_*`.

## Auth and session persistence

Models:

- `auth_accounts`;
- `account_communities`.

Services:

- `src/wtg/services/auth.py`;
- `src/wtg/services/admin_control.py`;
- `src/wtg/services/vk_user_token.py`.

Session files:

- default account: `WTG_STATE_ROOT/auth/current`;
- additional accounts: `WTG_STATE_ROOT/auth/accounts/<key>`.

Browser auth and API auth are separate. API polling capacity only counts
accounts with valid API auth state and canary `wall.get`.

## Admin jobs and worker queue

Core files:

- `src/wtg/services/admin_control.py`;
- `AdminJobWorker._claim_next_job`;
- `AdminJobWorker._execute_job`;
- `JOB_TYPES`;
- `JOB_STAGE_DEFINITIONS`.

Main job families:

- crawl/refresh: `crawl_all_groups`, `crawl_single_group`,
  `refresh_single_community`, `refresh_community_api`,
  `refresh_community_market`, `refresh_community_posts`,
  `refresh_all_group_posts`, `refresh_all_community_market`,
  `refresh_single_post`, `clear_community_profile`, `delete_community_posts`;
- API: `account_initial_sync`, `api_groups_sync`, `api_posts_sync`,
  `api_posts_force_sync`, `api_addresses_sync`, `api_products_sync`,
  `api_services_sync`, `api_rate_probe`, `api_token_refresh`,
  `api_fleet_pool_pass`;
- processing: `content_pipeline_run`, `full_account_pipeline`,
  `gold_posts_autolabel`, `event_topic_labels_autolabel`,
  `analyze_post`, `analyze_community`, `analyze_community_services`;
- auth: `reauthorize_account`, `validate_account_session`.

Visible jobs go to `auth_tool`; headless jobs go to `worker`.

## Crawler and VK API sync flows

Browser/API crawler implementation:

- `src/wtg/services/crawler.py`;
- `src/wtg/services/vk_parsers.py`;
- `src/wtg/services/post_media_storage.py`;
- `src/wtg/services/community_profile.py`;
- `src/wtg/services/community_market.py`;
- `src/wtg/services/community_services.py`.

API schedule implementation:

- `src/wtg/services/api_posts_schedule.py`;
- `api_log_settings.json` under runtime state;
- `api_schedule_steps` table.

Key flows:

- `account_initial_sync`: groups, anchors, addresses, products, services;
- `api_groups_sync`: VK groups list, account bindings, schedule rebuild;
- `api_posts_sync`: one due group, new posts/media, local pipeline for saved IDs;
- `api_products_sync` / `api_services_sync` / `api_addresses_sync`: best-effort
  free-slot updates;
- `api_rate_probe`: current limits/probe results;
- `api_token_refresh`: extracts/refreshes VK API user token state.

Production scope comes from VK account groups and explicit global pool sync, not
from `WTG_TARGET_COMMUNITIES`.

## Fast screen pool

Files:

- `src/wtg/services/fast_screen_pool.py`;
- `src/wtg/services/fast_wall_probe.py`;
- `src/wtg/cli/fast_screen_pool.py`.

Tables:

- `api_group_pool_items`;
- `fast_screen_pool_states`;
- `fast_screen_pool_checks`.

The pool syncs state rows for active global pool items, prioritizes groups with
recent event/promo material, leases due rows, probes wall content and writes
results. It can deactivate pool items by retention rules when reliable checks
show no suitable recent owner posts/materials.

## Content pipeline and OCR flow

Core files:

- `src/wtg/services/content_pipeline.py`;
- `src/wtg/services/ocr.py`;
- `src/wtg/services/ocr_features.py`;
- `src/wtg/services/ocr_jobs.py`;
- `src/wtg/cli/ocr_sidecar.py`;
- `src/wtg/cli/ocr_worker.py`.

Pipeline tables:

- `content_pipeline_runs`;
- `content_pipeline_posts`;
- `content_pipeline_items`;
- `content_pipeline_item_media`;
- `ocr_jobs`;
- `discovered_vk_groups`.

Current flow:

1. Select posts by account/group/explicit IDs/backlog.
2. Run deterministic local filter.
3. Run fast OCR preflight and local OCR cascade.
4. Queue/apply deep PaddleOCR for media-worthy/uncertain cases.
5. Run publication-intelligence rules.
6. Persist canonical publication decisions/materials.
7. Preserve legacy pipeline items as evidence/review layer.

`run_openrouter` remains optional and off unless explicitly enabled.

## Publication intelligence and canonical output

Package:

- `publication-intelligence/src/publication_intelligence/*`;
- CLI `pubintel`;
- docs under `publication-intelligence/docs`.

WTG integration:

- `src/wtg/services/publication_materials.py`;
- `src/wtg/services/publication_teacher.py`;
- `src/wtg/services/publication_intelligence_export.py`;
- `DOCS/publication-intelligence/README.md`.

Canonical tables:

- `publication_decisions`;
- `publication_materials`;
- `publication_material_media`;
- `publication_teacher_runs`;
- `publication_teacher_results`;
- `publication_teacher_applications`.

The legacy `publication_intelligence_shadow` payload key is still written for
UI/backward compatibility, but pipeline now also writes canonical
`publication_decisions` and `publication_materials`.

## Admin UI routes

FastAPI/Jinja app:

- `src/wtg/web/app.py`;
- templates: `apps/web/templates/admin`;
- static: `apps/web/static`.

Current pages:

- `/admin` -> `/admin/schedules`;
- `/admin/accounts`;
- `/admin/schedules`;
- `/admin/group-pool`;
- `/admin/accounts-vnc`;
- `/admin/accounts-vnc/operations`;
- `/admin/processing`;
- `/admin/results`;
- `/admin/logs`;
- `/admin/file-logs`;
- `/admin/groups`;
- `/admin/groups/{community_id}`;
- `/admin/market`;
- `/admin/labels`;
- `/admin/pipeline`;
- `/admin/pipeline/posts/{pipeline_post_id}`;
- `/admin/pipeline/items/{item_id}`;
- `/admin/posts/{post_id}`;
- `/admin/posts/{post_id}/local-check`.

Compatibility:

- `/review/*` redirects to `/admin/schedules`;
- `/admin/api-log*` redirects to schedule routes.

## DB and migrations

- SQLAlchemy models: `src/wtg/models/*`.
- DB sessions: `src/wtg/db/session.py`.
- Alembic env: `migrations/env.py`.
- Migrations: `migrations/versions/*.py`.

Latest migration: `0033_fast_screen_pool.py`.

Recent groups:

- `0025..0027`: API pool/fleet/browser-vnc;
- `0028..0029`: teacher and canonical publication output;
- `0030`: OCR jobs;
- `0031`: admin realtime indexes;
- `0032`: stale archive batches/items;
- `0033`: fast screen pool.

PostgreSQL is the integration boundary between web, worker, scheduler,
auth_tool, OCR worker, fast screen pool, reports and Telegram bot.

## Deploy/runtime layout

Production deploy files:

- `infra/deploy/compose.prod.yml`;
- `infra/deploy/deploy.sh`;
- `infra/deploy/prod.env.example`;
- `infra/deploy/nginx/wtg.conf`;
- `infra/deploy/systemd/*`.

Production runtime layout:

- `/opt/wtg/src/wtg`;
- `/opt/wtg/src/wtg-telegram-bot`;
- `/opt/wtg/state`;
- `/opt/wtg/deps`;
- `/opt/wtg/env/prod.env`;
- `/opt/wtg/bin/deploy.sh`.

Production compose starts `postgres`, `migrate`, `web`, `worker`,
`ocr_sidecar`, `ocr_worker`, `auth_tool`, `fast_screen_pool`, `scheduler` and
`telegram-bot`. `accounts_vnc` is a legacy profile service.

## Tests and acceptance commands

Use `uv` commands from the repository root:

```bash
uv run --extra dev pytest -q
uv run --extra dev pytest -q tests/unit/test_content_pipeline.py
uv run --extra dev pytest -q tests/unit/test_publication_intelligence.py tests/unit/test_publication_teacher.py
uv run --extra dev pytest -q tests/e2e/test_collection_admin_e2e.py -k 'pipeline or api_posts or full_account_pipeline'
git diff --check
```

Live VK tests are skipped by default:

```bash
WTG_RUN_LIVE_VK=1 uv run --extra dev pytest -q tests/live -m live_vk
```

Production verification targets:

```text
https://wtg.komaroff-dev.ru/admin/schedules
https://wtg.komaroff-dev.ru/admin/processing
https://wtg.komaroff-dev.ru/admin/results
https://wtg.komaroff-dev.ru/admin/pipeline?view=items&state=active&account_id=2
```

The URLs are protected by nginx Basic Auth. Credentials and deploy SSH material
must come from Vault `secret/codex/wtg/prod/*` and must not be printed.
