Purpose

Produce and persist the next concrete, scoped research tasks. PlannerAgent creates task dicts from mission + memory/graph + open hypotheses; TaskQueue stores them in SQLite, scores priority, enforces phase/status lifecycle, deduplicates, and hands the next pending task to the loop under hypothesis-terminal and duplicate-fingerprint constraints.

Source Files

Responsibilities

planner.py

• Hold static planning phases (_PLANNING_PHASES, 13 entries from scope_confirmation→report_generation, planner.py:39) and service→attack module map (_SERVICE_ATTACK_MAP, planner.py:57) covering ssh/smb/http/ftp/rdp/redis/… with module/tools/risk/priority.
• Offer plan_retry_with_modifications(failed_task, error, attempt, hypothesis_state) (planner.py:123): returns None for permanent errors or terminal hypotheses; otherwise rotates allowed_tools (→ alternative-tool:), doubles timeout_seconds on timeout, lowers priority by 5, tags investigation_method, and ensures fingerprint changes.
• Offer plan(mission, target_summary, graph_summary, open_hypotheses?, hypothesis_states?, existing_task_count, phase_filter, max_tasks) (planner.py:191): emits up to max_tasks (default 5) tasks across 5 buckets (scope confirmation when existing_count==0 → asset discovery → service identification → web/api mapping → per-hypothesis independent tool checks that avoid prior fingerprints) with filtering on objective/hypothesis presence, terminal-status exclusion, and seen (key,fingerprint) guard.
• Rank hypotheses (planner.py:388 rank_unresolved_hypotheses) by 45 + 20*uncertainty + 20*information_value + 10*confidence − 8*attempts − risk_penalty − 10*cost (risk_penalty: low 0, medium 10, high 25).

task_queue.py

• Define VALID_TASK_PHASES (recon/analysis/test/validate/exploit/post_exploit/report, task_queue.py:21) and _TASK_PHASE_ALIASES for legacy strings (task_queue.py:31).
• Persist tasks via create_task(task_data) (task_queue.py:55): calls HypothesisRepository.prepare_task for identity → validates phase/risk/status → computes _score_priority if priority is 0 → inserts tasks row with hypothesis_id/check_fingerprint.
• Serve get_next_task(target="") (task_queue.py:119): LEFT JOIN hypotheses WHERE tasks.status='pending' AND (h.id IS NULL OR h.status IN ('open','inconclusive')) ORDER BY priority DESC, created_at ASC — thereby never handing a terminal-hypothesis task.
• Support update_task_status/complete_task/block_task (task_queue.py:141), reset_stale_running() (task_queue.py:181 requeues running→pending on resume), get_task, list_open_tasks/list_blocked_tasks/list_completed_tasks (task_queue.py:216), deduplicate (deletes pending dup on target/objective/phase keeping lowest id, task_queue.py:269), reprioritize (rescues all pending via _score_priority), count_by_status/count_total.

Public Interfaces

planner.py

task_queue.py

Inputs/Outputs

State/Persistence

• tasks columns: phase, target, asset_type, objective, hypothesis, preconditions_json, allowed_tools_json, risk_level, priority, required_human_approval, success_criteria_json, stop_conditions_json, status ∈(pending,running,blocked,complete,failed,needs_approval), result_summary, block_reason, evidence_refs_json, hypothesis_id, check_fingerprint, created_at, updated_at.
• Indexes: idx_tasks_mission, idx_tasks_status(status,priority), idx_tasks_target, idx_tasks_hypothesis(mission_id,hypothesis_id,check_fingerprint).
• Priority 0..100 scored on insertion and on reprioritize():
  • Phase bonus: exploit 30 > validate 25 > test 20 > analysis 10 > recon 5.
  • Bonus on objective+hypothesis text: auth/bypass/privilege +10, idor/object +10, sensitive/data/leak +10, has hypothesis +5, confidence*10, expected_information_value*20.
  • Penalty: attempts*5, estimated_cost*10, risk high −10/medium −3, vague discover/scan without hypothesis −5, scan all −5.
• Dedup is predicate target+objective+phase duplicate (pending only, keeps lowest id).
• check_fingerprint + hypothesis_id come from outcome_judge.build_check_fingerprint / build_hypothesis_key via prepare_task.

Configuration

• PlannerAgent.risk_profile — currently stored, not yet branching phase creation (reserved).
• TaskQueue reads no config.yaml — behavior driven by caller-provided task_data.
• VALID_TASK_PHASES vs legacy aliases: unknown phases fall through to "test" (_normalize_task_phase).

Dependencies

• planner.py → outcome_judge.build_check_fingerprint, build_hypothesis_key, TERMINAL_HYPOTHESIS_STATUSES, HypothesisStatus
• task_queue.py → db.DatabaseManager, outcome_judge.HypothesisRepository, _new_id, _now_iso
• Callers: agent_loop.AgentLoop (primary), cli.cmd_next_task/cmd_list_tasks/cmd_run_task

Used By

• agent_loop.run: planner.plan + queue.create_task inside the “no pending tasks” branch; queue.get_next_task at top of every cycle; queue.reprioritize tail of cycle; planner.plan_retry_with_modifications on failed operational errors.
• cli.py: next-task, list-tasks, run-task [id].

Control Flow

flowchart TD
    A[AgentLoop.get_next_task] -->|found| B[scope/risk gate -> executor]
    A -->|None| C[planner.plan with memory+graph+hypotheses]
    C --> D{filter: has objective&hypothesis?\nterminal? fingerprint seen?}
    D -->|pass| E[queue.create_task -> emits task_created]
    E --> F{DuplicateInvestigationError / ClosedHypothesisError?}
    F -->|yes| G[emit task_rejected -> drop]
    F -->|no| H[queue.deduplicate]
    H --> I{created ==0?}
    I -->|yes && minima met| J[break loop]
    I -->|yes && unmet| K[stay alive with reason]
    I -->|no| A

Failure Modes

Invariants

• get_next_task never returns a task whose hypothesis is confirmed/refuted/exhausted.
• create_task is the sole writer of check_fingerprint; it always matches build_check_fingerprint(task_data).
• planner.plan never mutates DB — only returns dicts; Queue is the only persistence.
• reprioritize only touches status='pending' rows.
• reset_stale_running only transitions running → pending.

Security Boundaries

• No direct execution — planner/queue only produce and store task dicts; scope/risk gates are applied at execution time (next layer).
• hypothesis_id/check_fingerprint prevent unbounded retry loops on inconclusive hypotheses (max_inconclusive_attempts enforced in OutcomeJudge).

Tests

Run: python -m pytest tests/test_task_queue.py tests/test_agent_loop.py -v

Common Changes

Update This Document When

• PlannerAgent.plan buckets, plan_retry_with_modifications permanent-error list, or ranking formula change.
• TaskQueue schema (tasks phases/statuses, check_fingerprint handling) or _score_priority weights change.
• deduplicate / reprioritize / reset_stale_running semantics change.

Related Documentation

• docs/runtime-flows.md §Database-Backed Research Loop — planning ordering
• agent_loop.py (docs/components/flow-b/agent-loop.md) — consumer loop
• outcome_judge.py (docs/components/flow-b/observer-outcome.md) — hypothesis identity + fingerprint source
• tools/attack_modules/ (docs/attack-modules.md) — service modules that planner indirectly references via _SERVICE_ATTACK_MAP