Tuning & Evals

HEBBS doesn’t just store memories. It retrieves them using four strategies, each with configurable weights. The difference between a mediocre agent and a great one is tuning: measuring recall quality against your actual data and adjusting parameters until the agent finds the right information every time.

Why Tuning Matters

Out of the box, HEBBS uses default weights (0.5:0.2:0.2:0.1 for relevance, recency, importance, reinforcement) and defaults to similarity search. This works for general queries. But your domain has specific retrieval patterns:

• A sales agent needs recency-biased recall (recent calls matter more than old ones)
• A legal agent needs importance-biased recall (high-stakes clauses matter more than meeting notes)
• A support agent needs reinforcement-biased recall (frequently referenced solutions should surface first)

Tuning makes the agent’s memory match your domain.

Eval Results: How HEBBS Performs

We evaluate recall quality using curated datasets where correct answers are known. The key metric is precision@5: how many of the top 5 results are genuinely relevant.

Strategy Selection

Query Type	Similarity Only	HEBBS (Right Strategy)	Improvement
Temporal (“what happened before X?“)	42%	71%	+68%
Causal (“what caused X?“)	38%	62%	+63%
Analogical (“have we seen this pattern?“)	35%	50%	+43%
Similarity (“find docs about X”)	78%	79%	baseline
Overall (optimal strategy per query)	48%	66%	+37%

The biggest gains come from using the right strategy for the right question. Similarity search is the wrong tool for 60% of real-world agent queries.

Reflection Impact

After running the reflection pipeline:

Metric	Before	After	Change
Precision@5 (similarity)	78%	82%	+5%
Precision@5 (analogical)	50%	58%	+16%
Insights generated	0	142	new knowledge

Reflection consolidates raw memories into higher-level insights that match a broader range of queries.

How to Tune

Step 1: Profile Your Domain

What does your agent search for? Different domains weight strategies differently:

Domain	Primary Strategy	Weights (R:T:I:F)	Why
Sales/CRM	Temporal + Similarity	0.3:0.4:0.2:0.1	Recent interactions matter most
Legal	Similarity + Causal	0.4:0.1:0.4:0.1	Importance of clauses, precedent chains
Support	Similarity + Reinforcement	0.3:0.2:0.2:0.3	Frequently used solutions surface first
Engineering	Causal + Temporal	0.3:0.3:0.2:0.2	What happened and why
Research	Analogical + Similarity	0.5:0.1:0.2:0.2	Cross-domain pattern matching

Step 2: Write Evals

An eval is a query with a known correct answer. Write 10-20 evals that represent your real search patterns:

# Factual lookup
hebbs recall "What is our data retention policy?" --format json | jq '.results[0].content'
# Expected: content from policies/data-retention-v2.md

# Entity-scoped temporal
hebbs recall "What happened with Acme Corp last month?" --entity-id acme-corp --strategy temporal --format json
# Expected: recent call notes in chronological order

# Cross-entity analogical
hebbs recall "Which deals had similar budget pushback?" --strategy analogical --format json
# Expected: deals from other entities with budget reduction patterns

# Contradiction check
hebbs recall "What is Acme's budget?" --entity-id acme-corp --format json
# Expected: should surface both $100K and $50K with contradiction flag

Step 3: Run Baseline

Run your evals with default weights and record the results:

# Default weights
hebbs recall "your eval query" --weights 0.5:0.2:0.2:0.1 --format json

# Check: did the right answer appear in top 5?

Step 4: Tune Weights

Adjust weights based on what your domain needs:

# Recency-biased (for sales, support)
hebbs recall "your eval query" --weights 0.3:0.4:0.2:0.1

# Importance-biased (for legal, compliance)
hebbs recall "your eval query" --weights 0.3:0.1:0.5:0.1

# Reinforcement-biased (for support, repeated patterns)
hebbs recall "your eval query" --weights 0.2:0.2:0.2:0.4

Re-run your evals after each change. Compare precision@5 against the baseline.

Step 5: Tune Strategy Selection

Different queries need different strategies. Teach your agent (or your integration code) to select the right one:

async def smart_recall(hb, query, entity_id=None):
    """Select strategy based on query intent."""
    query_lower = query.lower()

    if any(w in query_lower for w in ["when", "timeline", "history", "happened"]):
        strategy = "temporal"
    elif any(w in query_lower for w in ["caused", "led to", "why did", "root cause"]):
        strategy = "causal"
    elif any(w in query_lower for w in ["similar", "pattern", "like this", "analogous"]):
        strategy = "analogical"
    else:
        strategy = "similarity"

    return await hb.recall(query, strategy=strategy, entity_id=entity_id)

Step 6: Run Reflection

After your knowledge base is populated, run reflection to generate insights:

hebbs reflect --entity-id acme-corp
hebbs insights --entity-id acme-corp --min-confidence 0.7

Insights improve recall quality by +5-16% because they provide higher-level summaries that match a broader range of queries. Run reflection weekly or after significant new content is uploaded.

Tuning via the Agent Skill

If your agent uses the HEBBS skill (Claude Code, OpenClaw), it can tune itself. Say:

tune my recall for this workspace

The skill will:

1. Profile your domain (asks you questions)
2. Generate evals from your actual content
3. Run baseline measurements
4. Test different weight configurations
5. Store the optimal weights as compiled rules

The compiled rules load automatically in every future conversation.

Example: CRM Tuning Session

You: tune recall for our sales workspace

Agent: I'll profile your workspace. What do your reps typically search for?

You: call history for specific accounts, competitive intel, case studies

Agent: Running baseline evals...
  - "Acme Corp call history" (temporal): baseline 3/5, tuned 5/5
  - "competitive comparison with Vendor X" (similarity): baseline 4/5, tuned 4/5
  - "deals with similar budget dynamics" (analogical): baseline 2/5, tuned 4/5

  Recommended weights: 0.3:0.4:0.2:0.1 (recency-biased)
  Overall precision: 48% → 72% (+50% improvement)

  Saving tuned parameters...

Key Takeaway

Default HEBBS is 37% better than similarity-only search. Tuned HEBBS, with the right weights and strategy selection for your domain, can improve recall precision by 50-70% over baseline. The difference compounds: better recall means better agent decisions, which means better outcomes for your business.