backstory/frontend/public/docs/resume-generation.md

The system follows a carefully designed pipeline with isolated stages to prevent fabrication:

System Architecture Overview

The system uses a pipeline of isolated analysis and generation steps:

1. Stage 1: Isolated Analysis

   • 1A: Job Analysis - Extracts requirements from job description only
   • 1B: Skill-Based Assessment - For each required skill, determine a Individisual Skill Assessment, adding it to a Skill Assessments Collection.

2. Stage 2: Resume Generation

   • Uses Skills Asessments Collection to generate a tailored resume.

3. Stage 3: Verification

   • Performs fact-checking to catch any remaining fabrications

flowchart TD
    subgraph "Stage 1: Isolated Analysis"
        subgraph "Stage 1A: Job Analysis"
            A1[Job Description Input] --> A2[Job Analysis LLM]
            A2 --> A3[Job Requirements JSON]
        end

        subgraph "Stage 1B: Skill-Based Assessment"
            B1[Resume Input] --> B2[Candidate Info]
            B2 --> B3[RAG System]
            A3 --> B4[Skill Assessment Generator]
            B3 --> B4
            B4 --> B5{For Each Required Skill}
            B5 --> B6[Skill-Focused LLM Query]
            B6 --> B7[Individual Skill Assessment]
            B7 --> B8[Skill Assessments Collection]
        end
    end
    
    subgraph "Stage 2: Resume Generation"
        C1[Skill Assessments Collection] --> C2[Resume Generator]
        C3[Original Resume Reference] --> C2
        C4[Candidate Information] --> C2
        C2 --> C5[Resume Generation Prompt]
        C5 --> C6[Resume Generation LLM]
        C6 --> C7[Tailored Resume Draft]
    end
    
    subgraph "Stage 3: Statistics & Verification"
        D1[Job Requirements JSON] --> D2[Match Statistics Calculator]
        D3[Skill Assessments Collection] --> D2
        D2 --> D4[Match Statistics]
        D4 --> D5[Verification LLM]
        C7 --> D5
        D5 --> D6{Verification Check}
        D6 -->|PASS| D7[Approved Resume]
        D6 -->|FAIL| D8[Correction Instructions]
        D8 --> C2
    end
    
    A3 --> B4
    B8 --> C1
    B8 --> D3
    B1 --> C3
    
    style A2 fill:#f9d77e,stroke:#333,stroke-width:2px
    style B6 fill:#f9d77e,stroke:#333,stroke-width:2px
    style C6 fill:#f9d77e,stroke:#333,stroke-width:2px
    style D5 fill:#f9d77e,stroke:#333,stroke-width:2px
    style B5 fill:#a3e4d7,stroke:#333,stroke-width:2px
    style D6 fill:#a3e4d7,stroke:#333,stroke-width:2px
    style D7 fill:#aed6f1,stroke:#333,stroke-width:2px
    style D8 fill:#f5b7b1,stroke:#333,stroke-width:2px

Stage 1: Isolated Analysis

1. Job Analysis: Extracts requirements from just the job description
2. Candidate Analysis: Catalogs qualifications for each job requirement from just the resume/context

Stage 2: Resume Generation

Creates a tailored resume using the skills collection and candidate information.

Stage 3: Verification

1. Performs fact-checking to catch any remaining fabrications
2. Corrects issues if needed and re-verifies

Key Anti-Fabrication Mechanisms

The system uses several techniques to prevent fabrication:

• Isolation of Analysis Stages: By analyzing the job and candidate separately, and having the LLM only provide evidence of a single skill per pass, the system prevents the LLM from prematurely creating connections that might lead to fabrication.
• Evidence Requirements: Each qualification included must have explicit evidence from the original materials.
• Conservative Transferability: The system is instructed to be conservative when claiming skills are transferable.
• Verification Layer: A dedicated verification step acts as a safety check to catch any remaining fabrications.
• Strict JSON Structures: Using structured JSON formats ensures information flows properly between stages.

Implementation Details

• Prompt Engineering: Each stage has carefully designed prompts with clear instructions and output formats.
• Error Handling: Comprehensive validation and error handling throughout the pipeline.
• Correction Loop: If verification fails, the system attempts to correct issues and re-verify.
• Traceability: Information in the final resume can be traced back to specific evidence in the original materials.