jketreno/backstory
1
0
Fork 0
Code Issues 12 Actions Releases Activity

Started adding Fact Check

Browse Source
This commit is contained in:
James Ketr 2025-04-08 18:53:36 -07:00
parent b3630ebba4
commit d3495983a1
5 changed files with 170 additions and 71 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
Dockerfile
View File

@ -330,7 +330,8 @@ WORKDIR /opt/ollama
# Download the nightly ollama release from ipex-llm
#ENV OLLAMA_VERSION=https://github.com/intel/ipex-llm/releases/download/v2.2.0-nightly/ollama-0.5.4-ipex-llm-2.2.0b20250226-ubuntu.tgz
ENV OLLAMA_VERSION=https://github.com/intel/ipex-llm/releases/download/v2.2.0-nightly/ollama-ipex-llm-2.2.0b20250313-ubuntu.tgz
#ENV OLLAMA_VERSION=https://github.com/intel/ipex-llm/releases/download/v2.2.0-nightly/ollama-ipex-llm-2.2.0b20250313-ubuntu.tgz
ENV OLLAMA_VERSION=https://github.com/intel/ipex-llm/releases/download/v2.2.0/ollama-ipex-llm-2.2.0-ubuntu.tgz
RUN wget -qO - ${OLLAMA_VERSION} | \
tar --strip-components=1 -C . -xzv

8
frontend/public/docs/about.md
View File

@ -18,16 +18,16 @@ The backstory about Backstory...
## This application was developed to achieve a few goals:
1. See if it is realistic to self-host AI LLMs. Turns out, it is -- with constraints. I've been meaning to write a blog post about what to buy to build an AI PC that can run the latest "small" (7B) parameter models.
2. Provide a recent example of my capabilities; many of my projects while working for Intel were internally facing. The source code to this project is available on [GitHub](https://github.com/jketreno/backstory). It doesn't touch on much of my history of work, however it does represent the pace at which I can adapt and develop useful solutions to fill a gap.
1. See if it is realistic to self-host AI LLMs. Turns out, it is -- with constraints. I don't have the GPU hardware to run models larger than about 8 billion parameters, which puts my local deployment in the realm of a Small Language Model (SLM.) I've been meaning to write a blog post about what to buy to build an AI PC that can run the latest "small" (7B) parameter models.
2. Provide a recent example of my capabilities; many of my projects while working for Intel were internally facing. The source code to this project is available on [GitHub](https://github.com/jketreno/backstory). It doesn't touch on much of my history of work, however it does represent the pace at which I can adapt and develop useful solutions to fill a gap. During this project's development I have had the opportunity to test and use many of the latest frontier models, which has allowed me to develop at a pace that far exceeds what I could have done even a year ago.
3. Explore Stable Diffusion (SD), Reinforced Learning (RL), Large Language Models (LLM), Paramater-Efficient Fine-Tuning (PEFT), Quantized Low-Rank Adapters (QLORA), open source and frontier models, tokenizers, and the vast open-source ecosystem for Machine Learning (ML) and Artificial Intelligence (AI). I wanted to do this to understand the strengths, weakness, and state of the industry in its development and deployment of those technologies.
4. My career at Intel was diverse. Over the years, I have worked on many projects almost everywhere in the computer ecosystem. That results in a resume that is either too long, or too short. This application is intended to provide a quick way for employers to ask the LLM about me. You can view my resume in totality, or use the Resume Builder to post your job position to see how I fit. Or go the Backstory and ask questions about the projects mentioned in my resume.
4. My career at Intel was diverse. Over the years, I have worked on many projects almost everywhere in the computer ecosystem. That results in a resume that is either too long, or too short. This application is intended to provide a quick way for employers to interactively ask about me. You can view my resume in totality, or use the Resume Builder to post your job position to see how I fit. Or go the Backstory and ask questions about the projects mentioned in my resume.
## Some questions I've been asked
Q. <ChatQuery text="Why aren't you providing this as a Platform As a Service (PaaS) application?"/>
A. I could; but I don't want to store your data. I also don't want to have to be on the hook for support of this service. I like it, it's fun, but it's not what I want as my day-gig, you know? If it was, I wouldn't have built this app...
A. I could; but I don't want to store your data. I also don't want to have to be on the hook for support of this service. I like it, it's fun, but it's not what I want as my day-gig, you know? If it was, I wouldn't be looking for a job...
Q. <ChatQuery text="Why can't I just ask Backstory these questions?"/>

2
frontend/src/App.tsx
View File

@ -47,7 +47,7 @@ import '@fontsource/roboto/700.css';
const welcomeMarkdown = `
# Welcome to Backstory
Backstory was written by James Ketrenos in order to provide answers to questions potential employers may have about his work history. In addition to being a RAG enabled expert system, the LLM has access to real-time data. You can ask things like:
Backstory was originally written by James Ketrenos in order to provide answers to questions potential employers may have about his work history. Now, you can deploy your own instance, host, and share your own Backstory. Backstory is a RAG enabled expert system with access to real-time data running self-hosted versions of industry leading Large and Small Language Models (LLM/SLMs). You can ask things like:
<ChatQuery text="What is James Ketrenos' work history?"/>
<ChatQuery text="What programming languages has James used?"/>

28
frontend/src/DocumentViewer.tsx
View File

@ -35,7 +35,9 @@ interface DocumentComponentProps {
interface DocumentViewerProps {
generateResume: (jobDescription: string) => void,
factCheck: (resume: string) => void,
resume: MessageData | undefined,
facts: MessageData | undefined,
sx?: SxProps<Theme>,
};
@ -59,7 +61,7 @@ const Document: React.FC<DocumentComponentProps> = ({ title, children }) => (
</Box>
);
const DocumentViewer: React.FC<DocumentViewerProps> = ({generateResume, resume, sx} : DocumentViewerProps) => {
const DocumentViewer: React.FC<DocumentViewerProps> = ({generateResume, factCheck, resume, facts, sx} : DocumentViewerProps) => {
const [jobDescription, setJobDescription] = useState<string>("");
const [processing, setProcessing] = useState<boolean>(false);
const theme = useTheme();
@ -139,7 +141,9 @@ const DocumentViewer: React.FC<DocumentViewerProps> = ({generateResume, resume,
placeholder="Paste a job description (or URL that resolves to one), then click Generate..."
/>
</Document>
<Button onClick={(e: any) => { triggerGeneration(jobDescription); }}>Generate</Button>
<Tooltip title="Generate">
<Button sx={{ m: 1, gap: 1 }} variant="contained" onClick={() => { triggerGeneration(jobDescription); }}>Generate<SendIcon /></Button>
</Tooltip>
</>) : (<Box sx={{ display: "flex", flexDirection: "column", overflow: "auto" }}>
<Document title="">{resume !== undefined && <Message message={resume} />}</Document>
{processing === true && <>
@ -160,9 +164,13 @@ const DocumentViewer: React.FC<DocumentViewerProps> = ({generateResume, resume,
<Typography>Generating resume...</Typography>
</Box>
</>}
<Card sx={{ display: "flex", overflow: "auto", minHeight: "fit-content", p: 1 }}>
{resume !== undefined || processing == true
? <Typography><b>NOTE:</b> As with all LLMs, hallucination is always a possibility. If the generated resume seems too good to be true, expand the <b>LLM information for this query</b> section (at the end of the resume) and click the links in the <b>Top RAG</b> matches to view the relavent RAG source document to read the details. Or go back to 'Backstory' and ask a question.</Typography>
<Card sx={{ display: "flex", overflow: "auto", minHeight: "fit-content", p: 1, flexDirection: "column" }}>
{facts !== undefined && <Message message={facts}/> }
{resume !== undefined || processing === true
? <>
<Typography><b>NOTE:</b> As with all LLMs, hallucination is always a possibility. If the generated resume seems too good to be true, <b>Fact Check</b> or, expand the <b>LLM information for this query</b> section (at the end of the resume) and click the links in the <b>Top RAG</b> matches to view the relavent RAG source document to read the details. Or go back to 'Backstory' and ask a question.</Typography> {processing === false && <Tooltip title="Fact Check">
<Button sx={{ m: 1, gap: 1 }} variant="contained" onClick={() => { resume && factCheck(resume.content); }}>Fact Check<SendIcon /></Button>
</Tooltip>}</>
: <Typography>Once you click <b>Generate</b> under the <b>Job Description</b>, a resume will be generated based on the user's RAG content and the job description.</Typography>
}
</Card>
@ -217,9 +225,13 @@ const DocumentViewer: React.FC<DocumentViewerProps> = ({generateResume, resume,
data-testid="loader"
/>
</Box>
<Card sx={{ display: "flex", overflow: "auto", minHeight: "fit-content", p: 1 }}>
{resume !== undefined || processing == true
? <Typography><b>NOTE:</b> As with all LLMs, hallucination is always a possibility. If the generated resume seems too good to be true, expand the <b>LLM information for this query</b> section (at the end of the resume) and click the links to the relavent RAG source document to read the details. Or go back to 'Backstory' and ask a question.</Typography>
<Card sx={{ display: "flex", overflow: "auto", minHeight: "fit-content", p: 1, flexDirection: "column" }}>
{facts !== undefined && <Message message={facts} />}
{resume !== undefined || processing === true
? <>
<Typography><b>NOTE:</b> As with all LLMs, hallucination is always a possibility. If the generated resume seems too good to be true, <b>Fact Check</b> or, expand the <b>LLM information for this query</b> section (at the end of the resume) and click the links in the <b>Top RAG</b> matches to view the relavent RAG source document to read the details. Or go back to 'Backstory' and ask a question.</Typography> { processing === false && <Tooltip title="Fact Check">
<Button sx={{ m: 1, gap: 1 }} variant="contained" onClick={() => { resume && factCheck(resume.content); }}>Fact Check<SendIcon /></Button>
</Tooltip>}</>
: <Typography>Once you click <b>Generate</b> under the <b>Job Description</b>, a resume will be generated based on the user's RAG content and the job description.</Typography>
}
</Card>

200
frontend/src/ResumeBuilder.tsx
View File

@ -1,12 +1,5 @@
import { useState, useCallback, useRef } from 'react';
import { useState, useCallback, } from 'react';
import Box from '@mui/material/Box';
import TextField from '@mui/material/TextField';
import PropagateLoader from "react-spinners/PropagateLoader";
import Tooltip from '@mui/material/Tooltip';
import Button from '@mui/material/Button';
import SendIcon from '@mui/icons-material/Send';
import { Message } from './Message';
import { SeverityType } from './Snack';
import { ContextStatus } from './ContextStatus';
import { MessageData } from './MessageMeta';
@ -22,15 +15,14 @@ interface ResumeBuilderProps {
setSnack: (message: string, severity?: SeverityType) => void,
};
const ResumeBuilder = ({scrollToBottom, isScrolledToBottom, setProcessing, processing, connectionBase, sessionId, setSnack} : ResumeBuilderProps) => {
const [jobDescription, setJobDescription] = useState<string>("");
const [generateStatus, setGenerateStatus] = useState<MessageData | undefined>(undefined);
const ResumeBuilder = ({ scrollToBottom, isScrolledToBottom, setProcessing, processing, connectionBase, sessionId, setSnack }: ResumeBuilderProps) => {
const [lastEvalTPS, setLastEvalTPS] = useState<number>(35);
const [lastPromptTPS, setLastPromptTPS] = useState<number>(430);
const [contextStatus, setContextStatus] = useState<ContextStatus>({ context_used: 0, max_context: 0 });
const [countdown, setCountdown] = useState<number>(0);
// const [countdown, setCountdown] = useState<number>(0);
const [resume, setResume] = useState<MessageData | undefined>(undefined);
const timerRef = useRef<any>(null);
const [facts, setFacts] = useState<MessageData | undefined>(undefined);
// const timerRef = useRef<any>(null);
const updateContextStatus = useCallback(() => {
fetch(connectionBase + `/api/context-status/${sessionId}`, {
@ -49,46 +41,38 @@ const ResumeBuilder = ({scrollToBottom, isScrolledToBottom, setProcessing, proce
});
}, [setContextStatus, connectionBase, setSnack, sessionId]);
const startCountdown = (seconds: number) => {
if (timerRef.current) clearInterval(timerRef.current);
setCountdown(seconds);
timerRef.current = setInterval(() => {
setCountdown((prev) => {
if (prev <= 1) {
clearInterval(timerRef.current);
timerRef.current = null;
if (isScrolledToBottom()) {
setTimeout(() => {
scrollToBottom();
}, 50)
}
return 0;
}
return prev - 1;
});
}, 1000);
};
// const startCountdown = (seconds: number) => {
// if (timerRef.current) clearInterval(timerRef.current);
// setCountdown(seconds);
// timerRef.current = setInterval(() => {
// setCountdown((prev) => {
// if (prev <= 1) {
// clearInterval(timerRef.current);
// timerRef.current = null;
// if (isScrolledToBottom()) {
// setTimeout(() => {
// scrollToBottom();
// }, 50)
// }
// return 0;
// }
// return prev - 1;
// });
// }, 1000);
// };
const stopCountdown = () => {
if (timerRef.current) {
clearInterval(timerRef.current);
timerRef.current = null;
setCountdown(0);
}
};
// const stopCountdown = () => {
// if (timerRef.current) {
// clearInterval(timerRef.current);
// timerRef.current = null;
// setCountdown(0);
// }
// };
if (sessionId === undefined) {
return (<></>);
}
const handleKeyPress = (event: any) => {
if (event.key === 'Enter' && !event.ctrlKey) {
generateResume(jobDescription);
}
};
const generateResume = async (jobDescription: string) => {
if (!jobDescription.trim()) return;
setResume(undefined);
@ -97,7 +81,7 @@ const ResumeBuilder = ({scrollToBottom, isScrolledToBottom, setProcessing, proce
setProcessing(true);
// Add initial processing message
setGenerateStatus({ role: 'assistant', content: 'Processing request...' });
//setGenerateStatus({ role: 'assistant', content: 'Processing request...' });
// Make the fetch request with proper headers
const response = await fetch(connectionBase + `/api/generate-resume/${sessionId}`, {
@ -114,7 +98,7 @@ const ResumeBuilder = ({scrollToBottom, isScrolledToBottom, setProcessing, proce
const estimate = Math.round(token_guess / lastEvalTPS + contextStatus.context_used / lastPromptTPS);
setSnack(`Job description sent. Response estimated in ${estimate}s.`, "info");
startCountdown(Math.round(estimate));
//startCountdown(Math.round(estimate));
if (!response.ok) {
throw new Error(`Server responded with ${response.status}: ${response.statusText}`);
@ -150,7 +134,7 @@ const ResumeBuilder = ({scrollToBottom, isScrolledToBottom, setProcessing, proce
// Force an immediate state update based on the message type
if (update.status === 'processing') {
// Update processing message with immediate re-render
setGenerateStatus({ role: 'info', content: update.message });
//setGenerateStatus({ role: 'info', content: update.message });
console.log(update.num_ctx);
// Add a small delay to ensure React has time to update the UI
@ -158,7 +142,7 @@ const ResumeBuilder = ({scrollToBottom, isScrolledToBottom, setProcessing, proce
} else if (update.status === 'done') {
// Replace processing message with final result
setGenerateStatus(undefined);
//setGenerateStatus(undefined);
setResume(update.message);
const metadata = update.message.metadata;
const evalTPS = metadata.eval_count * 10 ** 9 / metadata.eval_duration;
@ -168,7 +152,7 @@ const ResumeBuilder = ({scrollToBottom, isScrolledToBottom, setProcessing, proce
updateContextStatus();
} else if (update.status === 'error') {
// Show error
setGenerateStatus({ role: 'error', content: update.message });
//setGenerateStatus({ role: 'error', content: update.message });
}
} catch (e) {
setSnack("Error generating resume", "error")
@ -183,7 +167,7 @@ const ResumeBuilder = ({scrollToBottom, isScrolledToBottom, setProcessing, proce
const update = JSON.parse(buffer);
if (update.status === 'done') {
setGenerateStatus(undefined);
//setGenerateStatus(undefined);
setResume(update.message);
}
} catch (e) {
@ -191,14 +175,116 @@ const ResumeBuilder = ({scrollToBottom, isScrolledToBottom, setProcessing, proce
}
}
stopCountdown();
//stopCountdown();
setProcessing(false);
} catch (error) {
console.error('Fetch error:', error);
setSnack("Unable to process job description", "error");
setGenerateStatus({ role: 'error', content: `Error: ${error}` });
//setGenerateStatus({ role: 'error', content: `Error: ${error}` });
setProcessing(false);
stopCountdown();
//stopCountdown();
}
};
const factCheck = async (resume: string) => {
if (!resume.trim()) return;
setFacts(undefined);
try {
setProcessing(true);
const response = await fetch(connectionBase + `/api/fact-check/${sessionId}`, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Accept': 'application/json',
},
body: JSON.stringify({ content: resume.trim() }),
});
// We'll guess that the response will be around 500 tokens...
const token_guess = 500;
const estimate = Math.round(token_guess / lastEvalTPS + contextStatus.context_used / lastPromptTPS);
setSnack(`Resume sent for Fact Check. Response estimated in ${estimate}s.`, "info");
//startCountdown(Math.round(estimate));
if (!response.ok) {
throw new Error(`Server responded with ${response.status}: ${response.statusText}`);
}
if (!response.body) {
throw new Error('Response body is null');
}
// Set up stream processing with explicit chunking
const reader = response.body.getReader();
const decoder = new TextDecoder();
let buffer = '';
while (true) {
const { done, value } = await reader.read();
if (done) {
break;
}
const chunk = decoder.decode(value, { stream: true });
// Process each complete line immediately
buffer += chunk;
let lines = buffer.split('\n');
buffer = lines.pop() || ''; // Keep incomplete line in buffer
for (const line of lines) {
if (!line.trim()) continue;
try {
const update = JSON.parse(line);
// Force an immediate state update based on the message type
if (update.status === 'processing') {
// Add a small delay to ensure React has time to update the UI
await new Promise(resolve => setTimeout(resolve, 0));
} else if (update.status === 'done') {
// Replace processing message with final result
setFacts(update.message);
const metadata = update.message.metadata;
const evalTPS = metadata.eval_count * 10 ** 9 / metadata.eval_duration;
const promptTPS = metadata.prompt_eval_count * 10 ** 9 / metadata.prompt_eval_duration;
setLastEvalTPS(evalTPS ? evalTPS : 35);
setLastPromptTPS(promptTPS ? promptTPS : 35);
updateContextStatus();
} else if (update.status === 'error') {
}
} catch (e) {
setSnack("Error generating resume", "error")
console.error('Error parsing JSON:', e, line);
}
}
}
// Process any remaining buffer content
if (buffer.trim()) {
try {
const update = JSON.parse(buffer);
if (update.status === 'done') {
//setGenerateStatus(undefined);
setFacts(update.message);
}
} catch (e) {
setSnack("Error processing resume", "error")
}
}
//stopCountdown();
setProcessing(false);
} catch (error) {
console.error('Fetch error:', error);
setSnack("Unable to process resume", "error");
//setGenerateStatus({ role: 'error', content: `Error: ${error}` });
setProcessing(false);
//stopCountdown();
}
};
@ -211,7 +297,7 @@ const ResumeBuilder = ({scrollToBottom, isScrolledToBottom, setProcessing, proce
overflowY: "auto",
flexDirection: "column",
height: "calc(0vh - 0px)", // Hack to make the height work
}} {...{ generateResume, resume }} />
}} {...{ factCheck, facts, generateResume, resume }} />
</Box>
</Box>
);