Chat is working EXCEPT tools
This commit is contained in:
parent
759d859d18
commit
baaa6e8559
@ -1,3 +1,4 @@
|
||||
*
|
||||
!src
|
||||
!frontend
|
||||
**/node_modules
|
||||
|
||||
@ -317,6 +317,11 @@ ENV SYCL_CACHE_PERSISTENT=1
|
||||
ENV PATH=/opt/backstory:$PATH
|
||||
|
||||
COPY /src/ /opt/backstory/src/
|
||||
COPY /frontend/ /opt/backstory/frontend/
|
||||
|
||||
WORKDIR /opt/backstory/frontend
|
||||
RUN npm install --force
|
||||
WORKDIR /opt/backstory
|
||||
|
||||
ENTRYPOINT [ "/entrypoint.sh" ]
|
||||
|
||||
|
||||
@ -1,63 +0,0 @@
|
||||
# Plant Conservation Specialist
|
||||
|
||||
**Organization:** Oregon Botanical Gardens
|
||||
**Location:** Portland, Oregon
|
||||
**Duration:** April 2017 - May 2020
|
||||
|
||||
## Position Overview
|
||||
As Plant Conservation Specialist at the Oregon Botanical Gardens, I managed the institution's ex-situ conservation program for rare and endangered plant species native to the Pacific Northwest. This position bridged scientific research, hands-on horticulture, and public education.
|
||||
|
||||
## Key Responsibilities
|
||||
|
||||
### Ex-situ Conservation Program
|
||||
- Coordinated conservation collections for 45 rare and endangered plant species
|
||||
- Developed and maintained comprehensive database of accession records, phenology data, and propagation histories
|
||||
- Established genetic management protocols to ensure maximum diversity in conservation collections
|
||||
- Collaborated with Center for Plant Conservation on national rare plant conservation initiatives
|
||||
|
||||
### Propagation & Cultivation
|
||||
- Designed specialized growing environments for challenging species with specific habitat requirements
|
||||
- Experimented with various propagation techniques including tissue culture, specialized seed treatments, and vegetative methods
|
||||
- Maintained detailed documentation of successful and unsuccessful propagation attempts
|
||||
- Achieved first-ever successful cultivation of three critically endangered Oregon wildflowers
|
||||
|
||||
### Reintroduction Planning
|
||||
- Collaborated with federal and state agencies on plant reintroduction strategies
|
||||
- Conducted site assessments to evaluate habitat suitability for reintroductions
|
||||
- Developed monitoring protocols to track survival and reproduction of reintroduced populations
|
||||
- Prepared detailed reintroduction plans for 8 endangered species
|
||||
|
||||
### Research Projects
|
||||
- Designed and implemented germination studies for 15 rare species with unknown propagation requirements
|
||||
- Conducted pollination biology investigations for several endangered plant species
|
||||
- Collaborated with university researchers on seed viability and longevity studies
|
||||
- Maintained comprehensive records of phenological patterns across multiple growing seasons
|
||||
|
||||
### Education & Outreach
|
||||
- Developed educational materials explaining the importance of plant conservation
|
||||
- Led specialized tours focusing on rare plant conservation for visitors and donors
|
||||
- Trained volunteers in proper care of sensitive plant collections
|
||||
- Created interpretive signage for conservation garden displays
|
||||
|
||||
## Notable Projects
|
||||
|
||||
1. **Willamette Valley Prairie Species Recovery**
|
||||
- Established seed bank of 25 declining prairie species
|
||||
- Developed germination protocols that improved propagation success from 30% to 75%
|
||||
- Produced over 5,000 plants for restoration projects throughout the region
|
||||
|
||||
2. **Alpine Rare Plant Conservation Initiative**
|
||||
- Created specialized growing facilities mimicking alpine conditions
|
||||
- Successfully propagated 8 high-elevation rare species never before cultivated
|
||||
- Documented critical temperature and moisture requirements for germination
|
||||
|
||||
3. **Serpentine Soils Conservation Collection**
|
||||
- Developed custom soil mixes replicating challenging serpentine conditions
|
||||
- Maintained living collection of 12 rare serpentine endemic species
|
||||
- Created public display educating visitors about specialized plant adaptations
|
||||
|
||||
## Achievements
|
||||
- Received "Conservation Innovation Award" from the American Public Gardens Association (2019)
|
||||
- Developed propagation protocol for Kincaid's lupine that doubled germination success rates
|
||||
- Established Oregon Botanical Gardens' first dedicated conservation nursery facility
|
||||
- Created seed banking protocols adopted by three other botanical institutions
|
||||
@ -1,75 +0,0 @@
|
||||
# Research Assistant
|
||||
|
||||
**Organization:** Institute for Applied Ecology
|
||||
**Location:** Corvallis, Oregon
|
||||
**Duration:** January 2015 - March 2017
|
||||
|
||||
## Position Overview
|
||||
As Research Assistant at the Institute for Applied Ecology, I supported multiple research projects focused on native plant ecology and restoration techniques. This position provided foundational experience in applying scientific methods to practical conservation challenges.
|
||||
|
||||
## Key Responsibilities
|
||||
|
||||
### Field Surveys
|
||||
- Conducted comprehensive botanical surveys in diverse ecosystems throughout western Oregon
|
||||
- Documented population sizes, health metrics, and habitat conditions for threatened plant species
|
||||
- Established long-term monitoring plots using standardized protocols
|
||||
- Collected voucher specimens for herbarium collections following strict ethical guidelines
|
||||
- Mapped plant populations using GPS and GIS technologies
|
||||
|
||||
### Greenhouse Operations
|
||||
- Assisted with propagation of native plants for restoration experiments and projects
|
||||
- Maintained detailed records of seed treatments, germination rates, and growth parameters
|
||||
- Implemented and monitored experimental growing conditions for research projects
|
||||
- Managed irrigation systems and pest control for approximately 10,000 plants
|
||||
- Prepared plant materials for outplanting at restoration sites
|
||||
|
||||
### Data Collection & Analysis
|
||||
- Collected vegetation data using quadrat, transect, and plot-based sampling methods
|
||||
- Processed and organized large datasets for long-term monitoring studies
|
||||
- Performed statistical analyses using R to assess restoration treatment effectiveness
|
||||
- Created data visualization graphics for reports and publications
|
||||
- Maintained research databases ensuring data quality and accessibility
|
||||
|
||||
### Research Projects
|
||||
- **Prairie Restoration Techniques:**
|
||||
- Compared effectiveness of different site preparation methods on native plant establishment
|
||||
- Monitored post-treatment recovery of native species diversity
|
||||
- Documented invasive species response to various control techniques
|
||||
|
||||
- **Rare Plant Demography:**
|
||||
- Tracked population dynamics of three endangered Willamette Valley plant species
|
||||
- Monitored individual plant survival, growth, and reproductive output
|
||||
- Assessed impacts of management interventions on population trends
|
||||
|
||||
- **Seed Viability Studies:**
|
||||
- Tested germination requirements for 30+ native species
|
||||
- Evaluated effects of smoke, scarification, and stratification on dormancy
|
||||
- Documented optimal storage conditions for maintaining seed viability
|
||||
|
||||
### Publication Support
|
||||
- Co-authored three peer-reviewed publications on prairie restoration techniques
|
||||
- Prepared figures, tables, and data appendices for manuscripts
|
||||
- Conducted literature reviews on specialized ecological topics
|
||||
- Assisted with manuscript revisions based on peer review feedback
|
||||
|
||||
## Key Projects
|
||||
|
||||
1. **Willamette Valley Wet Prairie Restoration**
|
||||
- Implemented experimental plots testing 4 restoration techniques
|
||||
- Collected 3 years of post-treatment vegetation data
|
||||
- Documented successful establishment of 15 target native species
|
||||
|
||||
2. **Endangered Butterfly Habitat Enhancement**
|
||||
- Propagated host and nectar plants for Fender's blue butterfly habitat
|
||||
- Monitored plant-insect interactions in restoration sites
|
||||
- Assessed habitat quality improvements following restoration treatments
|
||||
|
||||
3. **Native Seed Production Research**
|
||||
- Tested cultivation methods for improving seed yields of 10 native species
|
||||
- Documented pollination requirements for optimal seed production
|
||||
- Developed harvest timing recommendations based on seed maturation patterns
|
||||
|
||||
## Publications
|
||||
- Johnson, T., **Morgan, E.**, et al. (2016). "Comparative effectiveness of site preparation techniques for prairie restoration." *Restoration Ecology*, 24(4), 472-481.
|
||||
- Williams, R., **Morgan, E.**, & Smith, B. (2016). "Germination requirements of Willamette Valley wet prairie species." *Native Plants Journal*, 17(2), 99-112.
|
||||
- **Morgan, E.**, Johnson, T., & Davis, A. (2017). "Long-term vegetation response to restoration treatments in degraded oak savanna." *Northwest Science*, 91(1), 27-39.
|
||||
@ -1,55 +0,0 @@
|
||||
# Senior Restoration Botanist
|
||||
|
||||
**Organization:** Pacific Northwest Conservation Alliance
|
||||
**Location:** Portland, Oregon
|
||||
**Duration:** June 2020 - Present
|
||||
|
||||
## Position Overview
|
||||
As Senior Restoration Botanist at the Pacific Northwest Conservation Alliance, I lead complex restoration projects aimed at preserving endangered plant communities throughout the Cascade Range. This role combines technical botanical expertise with project management and leadership responsibilities.
|
||||
|
||||
## Key Responsibilities
|
||||
|
||||
### Project Leadership
|
||||
- Design and implement comprehensive restoration plans for degraded ecosystems with emphasis on rare plant conservation
|
||||
- Lead field operations across multiple concurrent restoration sites covering over 2,000 acres
|
||||
- Establish measurable success criteria and monitoring protocols for all restoration projects
|
||||
- Conduct regular site assessments to track progress and adapt management strategies
|
||||
|
||||
### Native Plant Propagation
|
||||
- Oversee native plant nursery operations producing 75,000+ plants annually
|
||||
- Develop specialized propagation protocols for difficult-to-grow rare species
|
||||
- Maintain detailed records of germination rates, growth metrics, and treatment effects
|
||||
- Coordinate seed collection expeditions throughout diverse ecosystems of the Pacific Northwest
|
||||
|
||||
### Team Management
|
||||
- Supervise a core team of 5 field botanists and up to 12 seasonal restoration technicians
|
||||
- Conduct staff training on plant identification, restoration techniques, and field safety
|
||||
- Facilitate weekly team meetings and monthly progress reviews
|
||||
- Mentor junior staff and provide professional development opportunities
|
||||
|
||||
### Funding & Partnerships
|
||||
- Secured $750,000 in grant funding for riparian habitat restoration projects
|
||||
- Authored major sections of successful proposals to state and federal agencies
|
||||
- Manage project budgets ranging from $50,000 to $250,000
|
||||
- Cultivate partnerships with government agencies, tribes, and conservation NGOs
|
||||
|
||||
### Notable Projects
|
||||
1. **Willamette Valley Prairie Restoration Initiative**
|
||||
- Restored 350 acres of native prairie habitat
|
||||
- Reintroduced 12 threatened plant species with 85% establishment success
|
||||
- Developed innovative seeding techniques that increased native diversity by 40%
|
||||
|
||||
2. **Mount Hood Meadow Rehabilitation**
|
||||
- Led post-wildfire recovery efforts in alpine meadow ecosystems
|
||||
- Implemented erosion control measures using native plant materials
|
||||
- Achieved 90% reduction in invasive species cover within treatment areas
|
||||
|
||||
3. **Columbia River Gorge Rare Plant Recovery**
|
||||
- Established new populations of 5 federally listed plant species
|
||||
- Developed habitat suitability models to identify optimal reintroduction sites
|
||||
- Created monitoring protocols adopted by multiple conservation organizations
|
||||
|
||||
## Achievements
|
||||
- Received Excellence in Ecological Restoration Award from the Society for Ecological Restoration, Northwest Chapter (2023)
|
||||
- Featured in Oregon Public Broadcasting documentary on native plant conservation (2022)
|
||||
- Published 2 peer-reviewed articles on restoration techniques developed during project work
|
||||
@ -1,78 +1,272 @@
|
||||
Resume - Eliza Morgan
|
||||
# JAMES KETRENOS
|
||||
software architect, designer, developer, and team lead
|
||||
Beaverton, OR 97003
|
||||
|
||||
Professional Profile
|
||||
Dedicated botanist with 8 years of experience in plant conservation and ecological restoration. Specializing in native plant propagation and habitat rehabilitation for endangered flora species.
|
||||
Contact Information
|
||||
james@ketrenos.com
|
||||
(503) 501 8281
|
||||
|
||||
Email: eliza.morgan@botanist.com
|
||||
Phone: (555) 782-3941
|
||||
Address: 427 Maple Street, Portland, OR 97205
|
||||
Seeking an opportunity to contribute to the advancement of energy efficient AI solutions,
|
||||
James is a driven problem solver, solution creator, technical leader, and skilled software
|
||||
developer focused on rapid, high-quality results, with an eye toward bringing solutions to
|
||||
the market.
|
||||
|
||||
## SUMMARY
|
||||
|
||||
Problem-solving: Trusted resource for executive leadership, able to identify opportunities to
|
||||
bridge technical gaps, adopt new technologies, and improve efficiency and quality for internal
|
||||
and external customers.
|
||||
|
||||
Proficient: Adept in compiled and interpreted languages, the software frameworks built around them,
|
||||
and front- and backend infrastructure. Leveraging deep and varied experience to quickly find
|
||||
solutions. Rapidly familiarizes and puts to use new and emerging technologies.
|
||||
|
||||
Experienced: 20+ years of experience as an end-to-end Linux software architect, team lead,
|
||||
developer, system administrator, and user. Working with teams to bring together technologies
|
||||
into existing ecosystems for a myriad of technologies.
|
||||
|
||||
Leader: Frequent project lead spanning all areas of development and phases of the product life
|
||||
cycle from pre-silicon to post launch support. Capable change agent and mentor, providing
|
||||
technical engineering guidance to multiple teams and organizations.
|
||||
|
||||
Communicates: Thrives on helping people solve problems, working to educate others to help them
|
||||
better understand problems and work toward solutions.
|
||||
|
||||
## RECENT HISTORY
|
||||
|
||||
2024-2025: Present
|
||||
|
||||
* Developed 'backstory'
|
||||
* An Large Language Model (LLM) pipeline allowing interactive queries about James' resume.
|
||||
* Utilizing both Retrieval-Augmented Generation (RAG) and fine-tuned approaches, questions asked about James will use information from his resume and portfolio for answers.
|
||||
* Includes a full-stack React web ui and backend.
|
||||
* While developing Backstory, both Hugging Face and Ollama were used.
|
||||
* While exploring ways to augment the reduced 7B parameter model, various prompts, RAG, and Parameter-Efficient Fine-Tuning (PEFT) via Quantized Low-Rank Adapter (QLORA) were used.
|
||||
* Languages: Python, C/C++, JavaScript, TypeScript, React, HTML5, CSS, NodeJS, Markdown, git
|
||||
* URL: https://github.com/jketreno/backstory
|
||||
* Developed 'ze-monitor'
|
||||
* A lightweight C++ Linux application leveraging Level Zero Sysman APIs to provide 'top' like device monitoring of Intel GPUs.
|
||||
* URL: https://github.com/jketreno/ze-monitor
|
||||
* Languages: C++, CMake, DEB, RPM, bash, git
|
||||
* Provided a new release of Eikona, targetting the latest Android APIs.
|
||||
* Tracked down the latest versions of various packages, updating to align with deprecated and new APIs.
|
||||
* Languages: React Expo, Android SDK, Java, BitKeeper
|
||||
|
||||
2018-2024: Intel® Graphics Software Staff Architect and Lead
|
||||
|
||||
* Redefined how Intel approaches graphics enabling on Linux to meet customer and product timelines.
|
||||
* Many internal facing presentations discussing goals, roadmap, and reasoning.
|
||||
* Developed internal tools to help managers better forecast and look into where engineering developers were working in alignment with business objectives.
|
||||
* Languages: DEB, RPM, GitHub Actions (GHA), HTML5, JavaScript, JIRA, SQL, NodeJS, Google Docs APIs
|
||||
* Spearheaded internal projects to prove out the developer and customer deployment experience when using Intel graphics products with PyTorch, working to ensure all ingredients are available and consumable for success (from kernel driver integration, runtime, framework integration, up to containerized Python workload solution deployment.)
|
||||
*
|
||||
* Focused on improving the customer experience for Intel graphics software for Linux in the data center, high-performance compute clusters, and end users. Worked with several teams and business units to close gaps, improve our software, documentation, and release methodologies.
|
||||
* Worked with hardware and firmware teams to scope and define architectural solutions for customer features.
|
||||
* Worked with teams to add telemetry data available from Intel GPUs for use in Prometheus, Grafana, and collectd. Developed dashboards and infrastructure to manage node deployments using GitHub Actions, runners, and Docker.
|
||||
|
||||
1998-2018: Open Source Software Architect and Lead
|
||||
|
||||
* Defined software architecture for handheld devices, tablets, Internet of Things, smart appliances, and emerging technologies. Key resource to executive staff to investigate emerging technologies and drive solutions to close existing gaps
|
||||
* James career at Intel has been diverse. His strongest skills are related to quickly ramping on technologies being utilized in the market, identifying gaps in existing solutions, and working with teams to close those gaps. He excels at adopting and fitting new technology trends as they materialize in the industry.
|
||||
|
||||
## PROLONGED HISTORY
|
||||
|
||||
The following are technical areas James has been an architect, team lead, and/or individual contributor:
|
||||
|
||||
* Linux release infrastructure overhaul: Identified bottlenecks in the CI/CD build pipeline, built proof-of-concept, and moved to production for generating releases of Intel graphics software (https://dgpu-docs.intel.com) as well as internal dashboards and infrastructure for tracking build and release pipelines. JavaScript, HTML, Markdown, RTD, bash/python, Linux packaging, Linux repositories, Linux OS release life cycles, sqlite3. Worked with multiple teams across Intel to meet Intel’s requirements for public websites as well as to integrate with existing build and validation methodologies while educating teams on tools and infrastructure available from the ecosystem (vs. roll-your-own).
|
||||
* Board Explorer: Web app targeting developer ecosystem to utilize new single board computers, providing quick access to board details, circuits, and programming information. Delivered as a pure front-end service (no backend required) https://board-explorer.github.io/board-explorer/#quark_mcu_dev_kit_d2000. Tight coordination with UX design team. JavaScript, HTML, CSS, XML, hardware specs, programming specs.
|
||||
* (internal) Travel Requisition: Internal HTML application and backend enabling internal organizations to request travel approval and a manager front end to track budgetary expenditures in order to determine approval/deny decisions. NodeJS, JavaScript, Polymer, SQL. Tight coordination with internal requirements providers and UX design teams.
|
||||
* Developer Journey: Web infrastructure allowing engineers to document DIY processes. Front end for parsing, viewing, and following projects. Back end for managing content submitted (extended markdown) including images, videos, and screencasts. Tight coordination with UX design team.
|
||||
* Robotics: Worked with teams aligning on a ROS (Robot OS) roadmap and alignment. Presented at Embedded Linux conference on the state of open source and robotics. LIDAR, Intel RealSense, opencv, python, C. Developed a robotic vision controlled stewart platform that could play the marble game labyrinth.
|
||||
* Moblin and MeeGo architect: Focused on overall software architecture as well as moving forward multi-touch and the industry shift to resolution independent applications; all in a time before smart phones as we know them today. Qt, HTML5, EFL.
|
||||
* Marblin: An HTML/WebGL graphical application simulating the 2D collision physics of marbles in a 3D rendered canvas.
|
||||
* Linux Kernel: Developed and maintained initial Intel Pro Wireless 2100, 2200, and 3945 drivers in the Linux kernel. C, Software Defined Radios, IEEE 802.11, upstream kernel driver, team lead for team that took over the Intel wireless drivers, internal coordination regarding technical and legal issues surrounding the wireless stack.
|
||||
* Open source at Intel: Built proof-of-concepts to illustrate to management the potential and opportunities for Intel by embracing open source and Linux.
|
||||
* Intel Intercast Technology: Team lead for Intel Intercast software for Windows. Worked with 3rd party companies to integrate the technology into their solutions.
|
||||
|
||||
|
||||
# Professional Projects
|
||||
|
||||
## 1995 - 1998: Intel Intercast Technology
|
||||
* OS: Microsoft Windows Application, WinTV
|
||||
* Languages: C++
|
||||
* Role: Team lead and software architect
|
||||
* Microsoft Media infrastructure
|
||||
* Windows kernel driver work
|
||||
* Worked with internal teams and external companies to expand compatible hardware and integrate with Windows
|
||||
* Integration of Internet Explorer via COM embedding into the Intercast Viewer
|
||||
|
||||
## 1999 - 2024: Linux evangelist
|
||||
* One of the initial members of Intel's Open Source Technology Center (OTC)
|
||||
* Worked across Intel organizational boundaries to educate teams on the benefits and working model of the Linux open source ecosystem
|
||||
* Deep understanding of licensing issues, political dynamics, community goals, and business needs
|
||||
* Frequent resource for executive management and teams looking to leverage open source software
|
||||
|
||||
## 2000 - 2001: COM on Linux Prototype
|
||||
* Distributed component object model
|
||||
* Languages: C++, STL, Flex, Yacc, Bison
|
||||
* Role: Team lead and architect
|
||||
* Evaluated key performance differences between Microsoft Component Object Model's (COM) IUnknown (QueryInterface, AddRef, Release) vs. the Component Object Request Broker Architecture (CORBA) for both in-process and distributed cross-process and remote communication.
|
||||
* Developed prototype tool-chain and functional code providing a Linux compatible implementation of COM
|
||||
|
||||
## 1998 - 2000: Intel Dot Station
|
||||
* Languages: Java, C
|
||||
* Designed and built a "visual lens" Java plugin for Netscape Navigator
|
||||
* Role: Software architect
|
||||
|
||||
## 2000 - 2002: Carrier Grade Linux
|
||||
* OS distribution work
|
||||
* Contributed to the Linux System Base specification
|
||||
* Role: Team lead and software architect working with internal and external collaborators
|
||||
|
||||
## 2004 - 2006: Intel Wireless Linux Kernel Driver
|
||||
* Languages: C
|
||||
* Authored original ipw2100, ipw2200, and ipw3945 Linux kernel drivers
|
||||
* Built IEEE 802.11 wireless subsystem
|
||||
* Hosted Wireless Birds-of-a-Feather talk at the Ottawa Linux Symposium
|
||||
* Maintained SourceForge web presence, IRC channel, and community
|
||||
|
||||
## 2015 - 2018: Robotics
|
||||
* Languages: C, Python, NodeJS
|
||||
* "Maker" blogs on developing a Stewart Platform
|
||||
*
|
||||
* Image recognition and tracking
|
||||
* Presented at Embedded Linux Conference
|
||||
|
||||
## 2012 - 2017: RT24 - crosswalk
|
||||
* Chromium based native web application host
|
||||
* Role: Team lead and software architect
|
||||
* Worked with WebGL, Web Assembly, Native Client (NaCl)
|
||||
* Several internal presentations at various corporate events
|
||||
|
||||
## 2007 - 2009: Moblin
|
||||
* Tablet targetting OS distribution
|
||||
* Role: Team lead and software architect and requirements
|
||||
* Technology evaluation: Cairo, EFL, GTK, Clutter
|
||||
* Languages: C, C++, OpenGL
|
||||
|
||||
## 2012 - Web Sys Info
|
||||
* W3C
|
||||
* Tizen Working Group
|
||||
|
||||
## 2007 - 2017: Marblin
|
||||
* An interactive graphical stress test of rendering contexts
|
||||
* Ported to each framework being used for OS development
|
||||
* Originally written in C and using Clutter, ported to WebGL and EFL
|
||||
|
||||
## 2009 - 2011: MeeGo
|
||||
* The merging of Linux Foundation's Moblin with Nokia's Maemo
|
||||
* Coordinated and worked across business groups at Intel and Nokia
|
||||
* Role: Team lead and software architect
|
||||
* Focused on:
|
||||
* Resolution independent user interfaces
|
||||
* Multi-touch enabling in X
|
||||
* Educated teams on the interface paradigm shift to "mobile first"
|
||||
* Presented at MeeGo Conference
|
||||
* Languages: C++, QT, HTML5
|
||||
|
||||
## Android on Intel
|
||||
|
||||
## 2011 - 2013: Tizen
|
||||
* Rendering framework: Enlightenment Foundation Library (EFL)
|
||||
* Focused on: API specifications
|
||||
* Languages: JavaScript, HTML, C
|
||||
|
||||
## 2019 - 2024: Intel Graphics Architect
|
||||
* Technologies: C, JavaScript, HTML5, React, Markdown, bash, GitHub, GitHub Actions, Docker, Clusters, Data Center, Machine Learning, git
|
||||
* Role:
|
||||
* Set strategic direction for working with open source ecosystem
|
||||
* Worked with hardware and software architects to plan, execute, and support features
|
||||
* Set strategic direction for overhauling the customer experience for Intel graphics on Linux
|
||||
|
||||
# Personal Projects
|
||||
|
||||
1995 - 2023: Photo Management Software
|
||||
* Languages: C, JavaScript, PHP, HTML5, CSS, Polymer, React, SQL
|
||||
* Role: Personal photo management software, including facial recognition
|
||||
* Image classification, clustering, and identity
|
||||
|
||||
2020 - 2025: Eikona Android App
|
||||
* OS: Android
|
||||
* Languages: Java, Expo, React
|
||||
* Role: Maintainer for Android port
|
||||
|
||||
2019 - 2023: Peddlers of Ketran
|
||||
* Languages: JavaScript, React, NodeJS, HTML5, CSS
|
||||
* Features: Audio, Video, and Text chat. Full game plus expansions.
|
||||
* Role: Self-hosted online multiplayer clone of Settlers of Catan
|
||||
|
||||
2025: Ze-Monitor
|
||||
* C++ utility leveraging Level Zero API to monitor GPUs
|
||||
* https://github.com/jketreno/ze-monitor
|
||||
|
||||
Education:
|
||||
* Studied computer science at University of California San Diego, Oregon State University, and Portland State University
|
||||
* In his senior year of completing a bachelors degree, James left college to work full time for Intel Corporation
|
||||
|
||||
JAMES KETRENOS
|
||||
software architect, designer, developer, and team lead
|
||||
Beaverton, OR 97003
|
||||
|
||||
james@ketrenos.com
|
||||
(503) 501 8281
|
||||
|
||||
Professional Summary
|
||||
|
||||
James Ketrenos is an experienced software architect, designer, developer, and team lead with over two decades of expertise in Linux-based systems.
|
||||
Leveraging his strong problem-solving skills and deep technical knowledge across various domains including LLMs (Large Language Models),
|
||||
RAG (Relevance-Augmented Generation), and AI development, James brings a unique blend of industry experience to the table. His latest
|
||||
projects include 'Backstory,' an interactive GPT application that offers potential employers with insights into candidates.
|
||||
|
||||
Skills
|
||||
|
||||
Languages: C, C++, Python, Assembly, HTML, CSS, JavaScript, Typescript, Java
|
||||
Continuous Integration/Continuous Deployment, Build and Release, and Packaging Systems spanning multiple project domains and infrastructures
|
||||
SDKs and APIs for Internal and External Products (W3C, Tizen, MeeGo)
|
||||
Large Language Model (LLM) evaluation, selection, and deployment
|
||||
Retrieval-Augmented Generation (RAG) including data curation, processing, and searching
|
||||
Fine-Tuning LLMs
|
||||
Extending LLMs with tool capabilities (llama3.2, qwen2.5)
|
||||
|
||||
Professional Experience
|
||||
|
||||
Backstory | Software Architect | 2025 - Present
|
||||
Developed an interactive LLM application that streamlines content generation and information processing for job candidates. Built a full-stack React web UI and backend using RAG (Retrieval-Augmented Generation) techniques.
|
||||
Utilized Python, C/C++, JavaScript, TypeScript, React, HTML5, CSS, NodeJS, Markdown, git to create the project.
|
||||
Contributed significantly in developing the project using both Retrieval-Augmented Generation (RAG) and fine-tuned approaches.
|
||||
|
||||
Intel Graphics Software Staff Architect and Lead | Software Engineer, Intel Corporation | 2018 - 2024
|
||||
Redefined how Intel approaches graphics enabling on Linux to meet customer and product timelines. Spearheaded internal projects to prove out the developer and customer deployment experience when using Intel graphics products on Linux.
|
||||
Worked closely with hardware and firmware teams to scope and define architectural solutions for features such as firmware deployment and telemetry data collection from Intel GPUs in the data center.
|
||||
Deployed internal LLM instances for developer access to those capabilities without exposing content to the cloud
|
||||
Architected, prototyped, built, and deployed build and release infrastructure that improved efficiency and developer insight
|
||||
Collaborated across various business units and geographies, integrating multiple development and validation methodologies aligned with executive objectives.
|
||||
Worked with hardware architects to plan, execute, and support features for Intel graphics software.
|
||||
Led multiple internal development projects involving customer-facing features for the Linux data center environment.
|
||||
|
||||
|
||||
Open Source Evangelist | Software Architect, Intel Corporation | 2000 - 2018
|
||||
* Set strategic direction for working within the open-source ecosystem as part of the Intel Open Source Technology Center (OTC). Educated teams on the benefits and working model of Linux.
|
||||
* Contributed to the development of various open-source software projects such as MeeGo and Marblin. Played a key role in defining software architecture for handheld devices, tablets, IoT, smart appliances, and emerging technologies.
|
||||
* Worked across Intel organizational boundaries to educate teams on Linux's benefits and working model.
|
||||
* Built proof-of-concepts to illustrate potential and opportunities for open source software adoption within the company. Contributed to Linux kernel driver work, including development of initial drivers for wireless hardware.
|
||||
* Frequent resource to help businesses understand various open source licensing models and how they can be used
|
||||
|
||||
Developer Journey Web Infrastructure Lead | Software Architect, Intel Corporation | 2004 - 2017
|
||||
* Designed and built a comprehensive web infrastructure for engineers to document DIY processes, including the use of React and Polymer frameworks. Led a team in managing content submitted via extended markdown formats such as images and videos.
|
||||
|
||||
Robotics Engineer | Software Architect, Intel Corporation | 1998 - 2003
|
||||
* Developed a robotic vision-controlled Stewart platform capable of playing the marble Labyrinth game. Worked on image recognition and tracking applications using Python and NodeJS.
|
||||
|
||||
Education
|
||||
University of Washington, Seattle
|
||||
Master of Science in Botany - 2015
|
||||
Bachelor of Science in Environmental Science - 2013
|
||||
Thesis: "Propagation Techniques for Endangered Willamette Valley Prairie Species"
|
||||
Professional Experience
|
||||
Senior Restoration Botanist
|
||||
Pacific Northwest Conservation Alliance — Portland, OR
|
||||
June 2020 - Present
|
||||
* Attended University of California San Diego, Oregon State University, and Portland State University
|
||||
* During senior year in college, James began working full time at Intel and stopped pursuing a degree.
|
||||
|
||||
Lead restoration projects for endangered plant communities in the Cascade Range
|
||||
Develop and implement protocols for native seed collection and propagation
|
||||
Manage a team of 5 field botanists and 12 seasonal restoration technicians
|
||||
Secured $750,000 in grant funding for riparian habitat restoration projects
|
||||
Personal Projects
|
||||
|
||||
Plant Conservation Specialist
|
||||
Oregon Botanical Gardens — Portland, OR
|
||||
April 2017 - May 2020
|
||||
Eikona Android App | 2016 - 2024
|
||||
* Developed Android port of Eikona application utilizing Java, Expo, and React technologies.
|
||||
* Maintains new releases targeting the latest Android APIs.
|
||||
|
||||
Coordinated ex-situ conservation program for 45 rare and endangered plant species
|
||||
Maintained detailed documentation of propagation techniques and germination rates
|
||||
Collaborated with regional agencies on plant reintroduction strategies
|
||||
Developed educational materials for public outreach on native plant conservation
|
||||
|
||||
Research Assistant
|
||||
Institute for Applied Ecology — Corvallis, OR
|
||||
January 2015 - March 2017
|
||||
|
||||
Conducted field surveys of threatened plant populations throughout western Oregon
|
||||
Assisted with greenhouse propagation of native plants for restoration projects
|
||||
Collected and analyzed data for long-term vegetation monitoring studies
|
||||
Co-authored three peer-reviewed publications on prairie restoration techniques
|
||||
|
||||
Technical Skills
|
||||
|
||||
Native plant identification and taxonomy
|
||||
Seed collection and propagation methods
|
||||
Vegetation monitoring and data analysis
|
||||
Habitat assessment and restoration planning
|
||||
GIS mapping (ArcGIS, QGIS)
|
||||
Floristic surveys and botanical inventories
|
||||
|
||||
Professional Certifications
|
||||
|
||||
Certified Ecological Restoration Practitioner (CERP) - 2019
|
||||
Wetland Delineation Certification - 2016
|
||||
Wilderness First Responder - 2018
|
||||
|
||||
Professional Affiliations
|
||||
|
||||
Society for Ecological Restoration
|
||||
Botanical Society of America
|
||||
Native Plant Society of Oregon, Board Member
|
||||
|
||||
Publications & Presentations
|
||||
|
||||
Morgan, E., et al. (2023). "Restoration success of reintroduced Kincaid's lupine populations." Journal of Plant Conservation.
|
||||
Morgan, E., & Johnson, T. (2021). "Seed dormancy patterns in Pacific Northwest prairie species." Restoration Ecology.
|
||||
"Climate adaptation strategies for rare plant species," Annual Conference of the Society for Ecological Restoration, 2022.
|
||||
|
||||
Languages
|
||||
|
||||
English (Native)
|
||||
Spanish (Intermediate)
|
||||
|
||||
References
|
||||
Available upon request
|
||||
System Administrator | 1995 - present
|
||||
* Maintains a small cluster of servers providing email, photo management, game servers, and generative AI deployments.
|
||||
|
||||
|
||||
162
frontend/package-lock.json
generated
162
frontend/package-lock.json
generated
@ -38,6 +38,7 @@
|
||||
"web-vitals": "^2.1.4"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@craco/craco": "^7.1.0",
|
||||
"@types/plotly.js": "^2.35.5"
|
||||
}
|
||||
},
|
||||
@ -1974,12 +1975,35 @@
|
||||
"integrity": "sha512-GpVkmM8vF2vQUkj2LvZmD35JxeJOLCwJ9cUkugyk2nuhbv3+mJvpLYYt+0+USMxE+oj+ey/lJEnhZw75x/OMcQ==",
|
||||
"peer": true
|
||||
},
|
||||
"node_modules/@craco/craco": {
|
||||
"version": "7.1.0",
|
||||
"resolved": "https://registry.npmjs.org/@craco/craco/-/craco-7.1.0.tgz",
|
||||
"integrity": "sha512-oRAcPIKYrfPXp9rSzlsDNeOaVtDiKhoyqSXUoqiK24jCkHr4T8m/a2f74yXIzCbIheoUWDOIfWZyRgFgT+cpqA==",
|
||||
"dev": true,
|
||||
"dependencies": {
|
||||
"autoprefixer": "^10.4.12",
|
||||
"cosmiconfig": "^7.0.1",
|
||||
"cosmiconfig-typescript-loader": "^1.0.0",
|
||||
"cross-spawn": "^7.0.3",
|
||||
"lodash": "^4.17.21",
|
||||
"semver": "^7.3.7",
|
||||
"webpack-merge": "^5.8.0"
|
||||
},
|
||||
"bin": {
|
||||
"craco": "dist/bin/craco.js"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=6"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"react-scripts": "^5.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@cspotcode/source-map-support": {
|
||||
"version": "0.8.1",
|
||||
"resolved": "https://registry.npmjs.org/@cspotcode/source-map-support/-/source-map-support-0.8.1.tgz",
|
||||
"integrity": "sha512-IchNf6dN4tHoMFIn/7OE8LWZ19Y6q/67Bmf6vnGREv8RSbBVb9LPJxEcnwrcwX6ixSvaiGoomAUvu4YSxXrVgw==",
|
||||
"optional": true,
|
||||
"peer": true,
|
||||
"devOptional": true,
|
||||
"dependencies": {
|
||||
"@jridgewell/trace-mapping": "0.3.9"
|
||||
},
|
||||
@ -1991,8 +2015,7 @@
|
||||
"version": "0.3.9",
|
||||
"resolved": "https://registry.npmjs.org/@jridgewell/trace-mapping/-/trace-mapping-0.3.9.tgz",
|
||||
"integrity": "sha512-3Belt6tdc8bPgAtbcmdtNJlirVoTmEb5e2gC94PnkwEW9jI6CAHUeoG85tjWP5WquqfavoMtMwiG4P926ZKKuQ==",
|
||||
"optional": true,
|
||||
"peer": true,
|
||||
"devOptional": true,
|
||||
"dependencies": {
|
||||
"@jridgewell/resolve-uri": "^3.0.3",
|
||||
"@jridgewell/sourcemap-codec": "^1.4.10"
|
||||
@ -4406,29 +4429,25 @@
|
||||
"version": "1.0.11",
|
||||
"resolved": "https://registry.npmjs.org/@tsconfig/node10/-/node10-1.0.11.tgz",
|
||||
"integrity": "sha512-DcRjDCujK/kCk/cUe8Xz8ZSpm8mS3mNNpta+jGCA6USEDfktlNvm1+IuZ9eTcDbNk41BHwpHHeW+N1lKCz4zOw==",
|
||||
"optional": true,
|
||||
"peer": true
|
||||
"devOptional": true
|
||||
},
|
||||
"node_modules/@tsconfig/node12": {
|
||||
"version": "1.0.11",
|
||||
"resolved": "https://registry.npmjs.org/@tsconfig/node12/-/node12-1.0.11.tgz",
|
||||
"integrity": "sha512-cqefuRsh12pWyGsIoBKJA9luFu3mRxCA+ORZvA4ktLSzIuCUtWVxGIuXigEwO5/ywWFMZ2QEGKWvkZG1zDMTag==",
|
||||
"optional": true,
|
||||
"peer": true
|
||||
"devOptional": true
|
||||
},
|
||||
"node_modules/@tsconfig/node14": {
|
||||
"version": "1.0.3",
|
||||
"resolved": "https://registry.npmjs.org/@tsconfig/node14/-/node14-1.0.3.tgz",
|
||||
"integrity": "sha512-ysT8mhdixWK6Hw3i1V2AeRqZ5WfXg1G43mqoYlM2nc6388Fq5jcXyr5mRsqViLx/GJYdoL0bfXD8nmF+Zn/Iow==",
|
||||
"optional": true,
|
||||
"peer": true
|
||||
"devOptional": true
|
||||
},
|
||||
"node_modules/@tsconfig/node16": {
|
||||
"version": "1.0.4",
|
||||
"resolved": "https://registry.npmjs.org/@tsconfig/node16/-/node16-1.0.4.tgz",
|
||||
"integrity": "sha512-vxhUy4J8lyeyinH7Azl1pdd43GJhZH/tP2weN8TntQblOY+A0XbT8DJk1/oCPuOOyg/Ja757rG0CgHcWC8OfMA==",
|
||||
"optional": true,
|
||||
"peer": true
|
||||
"devOptional": true
|
||||
},
|
||||
"node_modules/@turf/area": {
|
||||
"version": "7.2.0",
|
||||
@ -6799,6 +6818,20 @@
|
||||
"wrap-ansi": "^7.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/clone-deep": {
|
||||
"version": "4.0.1",
|
||||
"resolved": "https://registry.npmjs.org/clone-deep/-/clone-deep-4.0.1.tgz",
|
||||
"integrity": "sha512-neHB9xuzh/wk0dIHweyAXv2aPGZIVk3pLMe+/RNzINf17fe0OG96QroktYAUm7SM1PBnzTabaLboqqxDyMU+SQ==",
|
||||
"dev": true,
|
||||
"dependencies": {
|
||||
"is-plain-object": "^2.0.4",
|
||||
"kind-of": "^6.0.2",
|
||||
"shallow-clone": "^3.0.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=6"
|
||||
}
|
||||
},
|
||||
"node_modules/clsx": {
|
||||
"version": "2.1.1",
|
||||
"resolved": "https://registry.npmjs.org/clsx/-/clsx-2.1.1.tgz",
|
||||
@ -7246,6 +7279,25 @@
|
||||
"node": ">=10"
|
||||
}
|
||||
},
|
||||
"node_modules/cosmiconfig-typescript-loader": {
|
||||
"version": "1.0.9",
|
||||
"resolved": "https://registry.npmjs.org/cosmiconfig-typescript-loader/-/cosmiconfig-typescript-loader-1.0.9.tgz",
|
||||
"integrity": "sha512-tRuMRhxN4m1Y8hP9SNYfz7jRwt8lZdWxdjg/ohg5esKmsndJIn4yT96oJVcf5x0eA11taXl+sIp+ielu529k6g==",
|
||||
"dev": true,
|
||||
"dependencies": {
|
||||
"cosmiconfig": "^7",
|
||||
"ts-node": "^10.7.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=12",
|
||||
"npm": ">=6"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"@types/node": "*",
|
||||
"cosmiconfig": ">=7",
|
||||
"typescript": ">=3"
|
||||
}
|
||||
},
|
||||
"node_modules/country-regex": {
|
||||
"version": "1.1.0",
|
||||
"resolved": "https://registry.npmjs.org/country-regex/-/country-regex-1.1.0.tgz",
|
||||
@ -7256,8 +7308,7 @@
|
||||
"version": "1.1.1",
|
||||
"resolved": "https://registry.npmjs.org/create-require/-/create-require-1.1.1.tgz",
|
||||
"integrity": "sha512-dcKFX3jn0MpIaXjisoRvexIJVEKzaq7z2rZKxf+MSr9TkdmHmsU4m2lcLojrj/FHl8mk5VxMmYA+ftRkP/3oKQ==",
|
||||
"optional": true,
|
||||
"peer": true
|
||||
"devOptional": true
|
||||
},
|
||||
"node_modules/cross-spawn": {
|
||||
"version": "7.0.6",
|
||||
@ -8138,8 +8189,7 @@
|
||||
"version": "4.0.2",
|
||||
"resolved": "https://registry.npmjs.org/diff/-/diff-4.0.2.tgz",
|
||||
"integrity": "sha512-58lmxKSA4BNyLz+HHMUzlOEpg09FV+ev6ZMe3vJihgdxzgcwZ8VoEEPmALCZG9LmqfVoNMMKpttIYTVG6uDY7A==",
|
||||
"optional": true,
|
||||
"peer": true,
|
||||
"devOptional": true,
|
||||
"engines": {
|
||||
"node": ">=0.3.1"
|
||||
}
|
||||
@ -9886,6 +9936,15 @@
|
||||
"node": ">=8"
|
||||
}
|
||||
},
|
||||
"node_modules/flat": {
|
||||
"version": "5.0.2",
|
||||
"resolved": "https://registry.npmjs.org/flat/-/flat-5.0.2.tgz",
|
||||
"integrity": "sha512-b6suED+5/3rTpUBdG1gupIl8MPFCAMA0QXwmljLhvCUKcUvdE4gWky9zpuGCcXHOsz4J9wPGNWq6OKpmIzz3hQ==",
|
||||
"dev": true,
|
||||
"bin": {
|
||||
"flat": "cli.js"
|
||||
}
|
||||
},
|
||||
"node_modules/flat-cache": {
|
||||
"version": "3.2.0",
|
||||
"resolved": "https://registry.npmjs.org/flat-cache/-/flat-cache-3.2.0.tgz",
|
||||
@ -11907,6 +11966,18 @@
|
||||
"url": "https://github.com/sponsors/sindresorhus"
|
||||
}
|
||||
},
|
||||
"node_modules/is-plain-object": {
|
||||
"version": "2.0.4",
|
||||
"resolved": "https://registry.npmjs.org/is-plain-object/-/is-plain-object-2.0.4.tgz",
|
||||
"integrity": "sha512-h5PpgXkWitc38BBMYawTYMWJHFZJVnBquFE57xFpjB8pJFiF6gZ+bU+WyI/yqXiFR5mdLsgYNaPe8uao6Uv9Og==",
|
||||
"dev": true,
|
||||
"dependencies": {
|
||||
"isobject": "^3.0.1"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=0.10.0"
|
||||
}
|
||||
},
|
||||
"node_modules/is-potential-custom-element-name": {
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/is-potential-custom-element-name/-/is-potential-custom-element-name-1.0.1.tgz",
|
||||
@ -12104,6 +12175,15 @@
|
||||
"resolved": "https://registry.npmjs.org/isexe/-/isexe-2.0.0.tgz",
|
||||
"integrity": "sha512-RHxMLp9lnKHGHRng9QFhRCMbYAcVpn69smSGcq3f36xjgVVWThj4qqLbTLlq7Ssj8B+fIQ1EuCEGI2lKsyQeIw=="
|
||||
},
|
||||
"node_modules/isobject": {
|
||||
"version": "3.0.1",
|
||||
"resolved": "https://registry.npmjs.org/isobject/-/isobject-3.0.1.tgz",
|
||||
"integrity": "sha512-WhB9zCku7EGTj/HQQRz5aUQEUeoQZH2bWcltRErOpymJ4boYE6wL9Tbr23krRPSZ+C5zqNSrSw+Cc7sZZ4b7vg==",
|
||||
"dev": true,
|
||||
"engines": {
|
||||
"node": ">=0.10.0"
|
||||
}
|
||||
},
|
||||
"node_modules/istanbul-lib-coverage": {
|
||||
"version": "3.2.2",
|
||||
"resolved": "https://registry.npmjs.org/istanbul-lib-coverage/-/istanbul-lib-coverage-3.2.2.tgz",
|
||||
@ -13527,8 +13607,7 @@
|
||||
"version": "1.3.6",
|
||||
"resolved": "https://registry.npmjs.org/make-error/-/make-error-1.3.6.tgz",
|
||||
"integrity": "sha512-s8UhlNe7vPKomQhC1qFelMokr/Sc3AgNbso3n74mVPA5LTZwkB9NlXf4XPamLxJE8h0gh73rM94xvwRT2CVInw==",
|
||||
"optional": true,
|
||||
"peer": true
|
||||
"devOptional": true
|
||||
},
|
||||
"node_modules/makeerror": {
|
||||
"version": "1.0.12",
|
||||
@ -18703,6 +18782,18 @@
|
||||
"resolved": "https://registry.npmjs.org/setprototypeof/-/setprototypeof-1.2.0.tgz",
|
||||
"integrity": "sha512-E5LDX7Wrp85Kil5bhZv46j8jOeboKq5JMmYM3gVGdGH8xFpPWXUMsNrlODCrkoxMEeNi/XZIwuRvY4XNwYMJpw=="
|
||||
},
|
||||
"node_modules/shallow-clone": {
|
||||
"version": "3.0.1",
|
||||
"resolved": "https://registry.npmjs.org/shallow-clone/-/shallow-clone-3.0.1.tgz",
|
||||
"integrity": "sha512-/6KqX+GVUdqPuPPd2LxDDxzX6CAbjJehAAOKlNpqqUpAqPM6HeL8f+o3a+JsyGjn2lv0WY8UsTgUJjU9Ok55NA==",
|
||||
"dev": true,
|
||||
"dependencies": {
|
||||
"kind-of": "^6.0.2"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=8"
|
||||
}
|
||||
},
|
||||
"node_modules/shallow-copy": {
|
||||
"version": "0.0.1",
|
||||
"resolved": "https://registry.npmjs.org/shallow-copy/-/shallow-copy-0.0.1.tgz",
|
||||
@ -20156,8 +20247,7 @@
|
||||
"version": "10.9.2",
|
||||
"resolved": "https://registry.npmjs.org/ts-node/-/ts-node-10.9.2.tgz",
|
||||
"integrity": "sha512-f0FFpIdcHgn8zcPSbf1dRevwt047YMnaiJM3u2w2RewrB+fob/zePZcrOyQoLMMO7aBIddLcQIEK5dYjkLnGrQ==",
|
||||
"optional": true,
|
||||
"peer": true,
|
||||
"devOptional": true,
|
||||
"dependencies": {
|
||||
"@cspotcode/source-map-support": "^0.8.0",
|
||||
"@tsconfig/node10": "^1.0.7",
|
||||
@ -20200,8 +20290,7 @@
|
||||
"version": "8.3.4",
|
||||
"resolved": "https://registry.npmjs.org/acorn-walk/-/acorn-walk-8.3.4.tgz",
|
||||
"integrity": "sha512-ueEepnujpqee2o5aIYnvHU6C0A42MNdsIDeqy5BydrkuC5R1ZuUFnm27EeFJGoEHJQgn3uleRvmTXaJgfXbt4g==",
|
||||
"optional": true,
|
||||
"peer": true,
|
||||
"devOptional": true,
|
||||
"dependencies": {
|
||||
"acorn": "^8.11.0"
|
||||
},
|
||||
@ -20213,8 +20302,7 @@
|
||||
"version": "4.1.3",
|
||||
"resolved": "https://registry.npmjs.org/arg/-/arg-4.1.3.tgz",
|
||||
"integrity": "sha512-58S9QDqG0Xx27YwPSt9fJxivjYl432YCwfDMfZ+71RAqUrZef7LrKQZ3LHLOwCS4FLNBplP533Zx895SeOCHvA==",
|
||||
"optional": true,
|
||||
"peer": true
|
||||
"devOptional": true
|
||||
},
|
||||
"node_modules/tsconfig-paths": {
|
||||
"version": "3.15.0",
|
||||
@ -20737,8 +20825,7 @@
|
||||
"version": "3.0.1",
|
||||
"resolved": "https://registry.npmjs.org/v8-compile-cache-lib/-/v8-compile-cache-lib-3.0.1.tgz",
|
||||
"integrity": "sha512-wa7YjyUGfNZngI/vtK0UHAN+lgDCxBPCylVXGp0zu59Fz5aiGtNXaq3DhIov063MorB+VfufLh3JlF2KdTK3xg==",
|
||||
"optional": true,
|
||||
"peer": true
|
||||
"devOptional": true
|
||||
},
|
||||
"node_modules/v8-to-istanbul": {
|
||||
"version": "8.1.1",
|
||||
@ -21081,6 +21168,20 @@
|
||||
"node": ">=10.13.0"
|
||||
}
|
||||
},
|
||||
"node_modules/webpack-merge": {
|
||||
"version": "5.10.0",
|
||||
"resolved": "https://registry.npmjs.org/webpack-merge/-/webpack-merge-5.10.0.tgz",
|
||||
"integrity": "sha512-+4zXKdx7UnO+1jaN4l2lHVD+mFvnlZQP/6ljaJVb4SZiwIKeUnrT5l0gkT8z+n4hKpC+jpOv6O9R+gLtag7pSA==",
|
||||
"dev": true,
|
||||
"dependencies": {
|
||||
"clone-deep": "^4.0.1",
|
||||
"flat": "^5.0.2",
|
||||
"wildcard": "^2.0.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=10.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/webpack-sources": {
|
||||
"version": "3.2.3",
|
||||
"resolved": "https://registry.npmjs.org/webpack-sources/-/webpack-sources-3.2.3.tgz",
|
||||
@ -21267,6 +21368,12 @@
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/wildcard": {
|
||||
"version": "2.0.1",
|
||||
"resolved": "https://registry.npmjs.org/wildcard/-/wildcard-2.0.1.tgz",
|
||||
"integrity": "sha512-CC1bOL87PIWSBhDcTrdeLo6eGT7mCFtrg0uIJtqJUFyK+eJnzl8A1niH56uu7KMa5XFrtiV+AQuHO3n7DsHnLQ==",
|
||||
"dev": true
|
||||
},
|
||||
"node_modules/word-wrap": {
|
||||
"version": "1.2.5",
|
||||
"resolved": "https://registry.npmjs.org/word-wrap/-/word-wrap-1.2.5.tgz",
|
||||
@ -21718,8 +21825,7 @@
|
||||
"version": "3.1.1",
|
||||
"resolved": "https://registry.npmjs.org/yn/-/yn-3.1.1.tgz",
|
||||
"integrity": "sha512-Ux4ygGWsu2c7isFWe8Yu1YluJmqVhxqK2cLXNQA5AcC3QfbGNpM7fu0Y8b/z16pXLnFxZYvWhd3fhBY9DLmC6Q==",
|
||||
"optional": true,
|
||||
"peer": true,
|
||||
"devOptional": true,
|
||||
"engines": {
|
||||
"node": ">=6"
|
||||
}
|
||||
|
||||
@ -56,7 +56,7 @@
|
||||
]
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/plotly.js": "^2.35.5",
|
||||
"@craco/craco": "^0.0.0"
|
||||
"@craco/craco": "^7.1.0",
|
||||
"@types/plotly.js": "^2.35.5"
|
||||
}
|
||||
}
|
||||
|
||||
@ -255,11 +255,20 @@ const App = () => {
|
||||
icon: <SettingsIcon />
|
||||
},
|
||||
children: (
|
||||
<Box className="ChatBox">
|
||||
<Scrollable
|
||||
autoscroll={false}
|
||||
sx={{
|
||||
maxWidth: "1024px",
|
||||
height: "calc(100vh - 72px)",
|
||||
flexDirection: "column",
|
||||
margin: "0 auto",
|
||||
p: 1,
|
||||
}}
|
||||
>
|
||||
{sessionId !== undefined &&
|
||||
<Controls {...{ sessionId, setSnack, connectionBase }} />
|
||||
}
|
||||
</Box >
|
||||
</Scrollable>
|
||||
)
|
||||
}];
|
||||
}, [about, connectionBase, sessionId, setSnack, isMobile]);
|
||||
|
||||
@ -6,8 +6,10 @@ import Accordion from '@mui/material/Accordion';
|
||||
import AccordionSummary from '@mui/material/AccordionSummary';
|
||||
import AccordionDetails from '@mui/material/AccordionDetails';
|
||||
import ExpandMoreIcon from '@mui/icons-material/ExpandMore';
|
||||
import LocationSearchingIcon from '@mui/icons-material/LocationSearching';
|
||||
|
||||
import { MessageRoles } from './Message';
|
||||
import { ErrorOutline, InfoOutline, Memory, Psychology, /* Stream, */ } from '@mui/icons-material';
|
||||
|
||||
interface ChatBubbleProps {
|
||||
role: MessageRoles,
|
||||
@ -20,7 +22,7 @@ interface ChatBubbleProps {
|
||||
}
|
||||
|
||||
function ChatBubble(props: ChatBubbleProps) {
|
||||
const { role, isFullWidth, children, sx, className, title } = props;
|
||||
const { role, isFullWidth, children, sx, className, title }: ChatBubbleProps = props;
|
||||
const theme = useTheme();
|
||||
|
||||
const defaultRadius = '16px';
|
||||
@ -116,6 +118,28 @@ function ChatBubble(props: ChatBubbleProps) {
|
||||
lineHeight: '1.3', // More compact line height
|
||||
fontFamily: theme.typography.fontFamily, // Consistent font with your theme
|
||||
},
|
||||
'thinking': {
|
||||
...defaultStyle
|
||||
},
|
||||
'streaming': {
|
||||
...defaultStyle
|
||||
},
|
||||
'processing': {
|
||||
...defaultStyle
|
||||
},
|
||||
};
|
||||
styles["thinking"] = styles["status"]
|
||||
styles["streaming"] = styles["assistant"]
|
||||
styles["processing"] = styles["status"]
|
||||
|
||||
const icons: any = {
|
||||
"searching": <Memory />,
|
||||
"thinking": <Psychology />,
|
||||
// "streaming": <Stream />,
|
||||
"tooling": <LocationSearchingIcon />,
|
||||
"processing": <LocationSearchingIcon />,
|
||||
"error": <ErrorOutline color='error' />,
|
||||
"info": <InfoOutline color='info' />,
|
||||
};
|
||||
|
||||
if (role === 'content' && title) {
|
||||
@ -138,9 +162,13 @@ function ChatBubble(props: ChatBubbleProps) {
|
||||
);
|
||||
}
|
||||
|
||||
console.log(role);
|
||||
return (
|
||||
<Box className={className} sx={{ ...styles[role], ...sx }}>
|
||||
{children}
|
||||
<Box className={className} sx={{ ...(styles[role] !== undefined ? styles[role] : styles["status"]), gap: 1, display: "flex", ...sx, flexDirection: "row" }}>
|
||||
{icons[role] !== undefined && icons[role]}
|
||||
<Box sx={{ p: 0, m: 0, gap: 0, display: "flex", flexGrow: 1, flexDirection: "column" }}>
|
||||
{children}
|
||||
</Box>
|
||||
</Box>
|
||||
);
|
||||
}
|
||||
|
||||
@ -223,7 +223,7 @@ const Controls = ({ sessionId, setSnack, connectionBase }: ControlsParams) => {
|
||||
}, [systemInfo, setSystemInfo, connectionBase, setSnack, sessionId])
|
||||
|
||||
useEffect(() => {
|
||||
setEditSystemPrompt(systemPrompt);
|
||||
setEditSystemPrompt(systemPrompt.trim());
|
||||
}, [systemPrompt, setEditSystemPrompt]);
|
||||
|
||||
const toggleRag = async (tool: Tool) => {
|
||||
@ -314,39 +314,36 @@ const Controls = ({ sessionId, setSnack, connectionBase }: ControlsParams) => {
|
||||
|
||||
const handleKeyPress = (event: any) => {
|
||||
if (event.key === 'Enter' && event.ctrlKey) {
|
||||
switch (event.target.id) {
|
||||
case 'SystemPromptInput':
|
||||
setSystemPrompt(editSystemPrompt);
|
||||
break;
|
||||
}
|
||||
setSystemPrompt(editSystemPrompt);
|
||||
}
|
||||
};
|
||||
|
||||
return (<div className="Controls">
|
||||
<Typography component="span" sx={{ mb: 1 }}>
|
||||
You can change the information available to the LLM by adjusting the following settings:
|
||||
|
||||
</Typography>
|
||||
<Accordion>
|
||||
<AccordionSummary expandIcon={<ExpandMoreIcon />}>
|
||||
<Typography component="span">System Prompt</Typography>
|
||||
</AccordionSummary>
|
||||
<AccordionActions style={{ flexDirection: "column" }}>
|
||||
<AccordionActions style={{ display: "flex", flexDirection: "column" }}>
|
||||
<TextField
|
||||
variant="outlined"
|
||||
fullWidth
|
||||
multiline
|
||||
slotProps={{
|
||||
htmlInput: { style: { fontSize: "0.85rem", lineHeight: "1.25rem" } }
|
||||
}}
|
||||
type="text"
|
||||
value={editSystemPrompt}
|
||||
onChange={(e) => setEditSystemPrompt(e.target.value)}
|
||||
onKeyDown={handleKeyPress}
|
||||
placeholder="Enter the new system prompt.."
|
||||
id="SystemPromptInput"
|
||||
/>
|
||||
<div style={{ display: "flex", flexDirection: "row", gap: "8px", paddingTop: "8px" }}>
|
||||
<Button variant="contained" disabled={editSystemPrompt === systemPrompt} onClick={() => { setSystemPrompt(editSystemPrompt); }}>Set</Button>
|
||||
<Box sx={{ display: "flex", flexDirection: "row", gap: "8px", paddingTop: "8px" }}>
|
||||
<Button variant="contained" disabled={editSystemPrompt.trim() === systemPrompt.trim()} onClick={() => { setSystemPrompt(editSystemPrompt.trim()); }}>Set</Button>
|
||||
<Button variant="outlined" onClick={() => { reset(["system_prompt"], "System prompt reset."); }} color="error">Reset</Button>
|
||||
</div>
|
||||
</Box>
|
||||
</AccordionActions>
|
||||
</Accordion>
|
||||
|
||||
|
||||
@ -390,15 +390,6 @@ const Conversation = forwardRef<ConversationHandle, ConversationProps>(({
|
||||
const update = JSON.parse(line);
|
||||
|
||||
switch (update.status) {
|
||||
case 'searching':
|
||||
case 'processing':
|
||||
case 'thinking':
|
||||
// Force an immediate state update based on the message type
|
||||
// Update processing message with immediate re-render
|
||||
setProcessingMessage({ role: 'status', content: update.response });
|
||||
// Add a small delay to ensure React has time to update the UI
|
||||
await new Promise(resolve => setTimeout(resolve, 0));
|
||||
break;
|
||||
case 'done':
|
||||
console.log('Done processing:', update);
|
||||
// Replace processing message with final result
|
||||
@ -440,6 +431,13 @@ const Conversation = forwardRef<ConversationHandle, ConversationProps>(({
|
||||
setProcessingMessage(undefined);
|
||||
}, 5000);
|
||||
|
||||
// Add a small delay to ensure React has time to update the UI
|
||||
await new Promise(resolve => setTimeout(resolve, 0));
|
||||
break;
|
||||
default:
|
||||
// Force an immediate state update based on the message type
|
||||
// Update processing message with immediate re-render
|
||||
setProcessingMessage({ role: update.status, content: update.response });
|
||||
// Add a small delay to ensure React has time to update the UI
|
||||
await new Promise(resolve => setTimeout(resolve, 0));
|
||||
break;
|
||||
@ -516,7 +514,7 @@ const Conversation = forwardRef<ConversationHandle, ConversationProps>(({
|
||||
flexDirection: "column",
|
||||
alignItems: "center",
|
||||
justifyContent: "center",
|
||||
mb: 1,
|
||||
m: 1,
|
||||
}}>
|
||||
<PropagateLoader
|
||||
size="10px"
|
||||
|
||||
@ -28,7 +28,7 @@ import { VectorVisualizer } from './VectorVisualizer';
|
||||
import { SetSnackType } from './Snack';
|
||||
import { CopyBubble } from './CopyBubble';
|
||||
|
||||
type MessageRoles = 'info' | 'user' | 'assistant' | 'system' | 'status' | 'error' | 'content';
|
||||
type MessageRoles = 'info' | 'user' | 'assistant' | 'system' | 'status' | 'error' | 'content' | 'thinking' | 'processing';
|
||||
|
||||
type MessageData = {
|
||||
role: MessageRoles,
|
||||
@ -52,7 +52,9 @@ interface MessageMetaData {
|
||||
},
|
||||
origin: string,
|
||||
rag: any,
|
||||
tools: any[],
|
||||
tools?: {
|
||||
tool_calls: any[],
|
||||
},
|
||||
eval_count: number,
|
||||
eval_duration: number,
|
||||
prompt_eval_count: number,
|
||||
@ -151,7 +153,7 @@ const MessageMeta = (props: MessageMetaProps) => {
|
||||
</Accordion>
|
||||
}
|
||||
{
|
||||
tools !== undefined && tools.length !== 0 &&
|
||||
tools !== undefined && tools.tool_calls && tools.tool_calls.length !== 0 &&
|
||||
<Accordion sx={{ boxSizing: "border-box" }}>
|
||||
<AccordionSummary expandIcon={<ExpandMoreIcon />}>
|
||||
<Box sx={{ fontSize: "0.8rem" }}>
|
||||
@ -159,26 +161,24 @@ const MessageMeta = (props: MessageMetaProps) => {
|
||||
</Box>
|
||||
</AccordionSummary>
|
||||
<AccordionDetails>
|
||||
{tools.map((tool: any, index: number) => <Box key={index}>
|
||||
{index !== 0 && <Divider />}
|
||||
<Box sx={{ fontSize: "0.75rem", display: "flex", flexDirection: "column", mt: 0.5 }}>
|
||||
<div style={{ display: "flex", paddingRight: "1rem", whiteSpace: "nowrap" }}>
|
||||
{tool.tool}
|
||||
</div>
|
||||
<div style={{
|
||||
display: "flex",
|
||||
padding: "3px",
|
||||
whiteSpace: "pre-wrap",
|
||||
flexGrow: 1,
|
||||
border: "1px solid #E0E0E0",
|
||||
wordBreak: "break-all",
|
||||
maxHeight: "5rem",
|
||||
overflow: "auto"
|
||||
}}>
|
||||
{JSON.stringify(tool.result, null, 2)}
|
||||
</div>
|
||||
</Box>
|
||||
</Box>)}
|
||||
{
|
||||
tools.tool_calls.map((tool: any, index: number) =>
|
||||
<Box key={index} sx={{ m: 0, p: 1, pt: 0, display: "flex", flexDirection: "column", border: "1px solid #e0e0e0" }}>
|
||||
{index !== 0 && <Divider />}
|
||||
<Box sx={{ fontSize: "0.75rem", display: "flex", flexDirection: "column", mt: 1, mb: 1, fontWeight: "bold" }}>
|
||||
{tool.name}
|
||||
</Box>
|
||||
<JsonView displayDataTypes={false} objectSortKeys={true} collapsed={2} value={JSON.parse(tool.content)} style={{ fontSize: "0.8rem", maxHeight: "20rem", overflow: "auto" }}>
|
||||
<JsonView.String
|
||||
render={({ children, ...reset }) => {
|
||||
if (typeof (children) === "string" && children.match("\n")) {
|
||||
return <pre {...reset} style={{ display: "flex", border: "none", ...reset.style }}>{children}</pre>
|
||||
}
|
||||
}}
|
||||
/>
|
||||
</JsonView>
|
||||
</Box>)
|
||||
}
|
||||
</AccordionDetails>
|
||||
</Accordion>
|
||||
}
|
||||
@ -229,8 +229,16 @@ const MessageMeta = (props: MessageMetaProps) => {
|
||||
</AccordionSummary>
|
||||
<AccordionDetails>
|
||||
{typeof (value) === "string" ?
|
||||
<pre>{value}</pre> :
|
||||
<JsonView collapsed={1} value={value as any} style={{ fontSize: "0.8rem", maxHeight: "20rem", overflow: "auto" }} />
|
||||
<pre style={{ border: "none", margin: 0, padding: 0 }}>{value}</pre> :
|
||||
<JsonView displayDataTypes={false} objectSortKeys={true} collapsed={2} value={value as any} style={{ fontSize: "0.8rem", maxHeight: "20rem", overflow: "auto" }}>
|
||||
<JsonView.String
|
||||
render={({ children, ...reset }) => {
|
||||
if (typeof (children) === "string" && children.match("\n")) {
|
||||
return <pre {...reset} style={{ display: "flex", border: "none", ...reset.style }}>{children}</pre>
|
||||
}
|
||||
}}
|
||||
/>
|
||||
</JsonView>
|
||||
}
|
||||
</AccordionDetails>
|
||||
</Accordion>
|
||||
|
||||
233
src/server.py
233
src/server.py
@ -18,6 +18,8 @@ import math
|
||||
import warnings
|
||||
from typing import Any
|
||||
|
||||
from uuid import uuid4
|
||||
|
||||
def try_import(module_name, pip_name=None):
|
||||
try:
|
||||
__import__(module_name)
|
||||
@ -28,7 +30,6 @@ def try_import(module_name, pip_name=None):
|
||||
# Third-party modules with import checks
|
||||
try_import("ollama")
|
||||
try_import("requests")
|
||||
try_import("bs4", "beautifulsoup4")
|
||||
try_import("fastapi")
|
||||
try_import("uvicorn")
|
||||
try_import("numpy")
|
||||
@ -37,7 +38,6 @@ try_import("sklearn")
|
||||
|
||||
import ollama
|
||||
import requests
|
||||
from bs4 import BeautifulSoup
|
||||
from contextlib import asynccontextmanager
|
||||
from fastapi import FastAPI, Request, BackgroundTasks
|
||||
from fastapi.responses import JSONResponse, StreamingResponse, FileResponse, RedirectResponse
|
||||
@ -49,17 +49,11 @@ from sklearn.preprocessing import MinMaxScaler
|
||||
|
||||
from utils import (
|
||||
rag as Rag,
|
||||
tools as Tools,
|
||||
Context, Conversation, Message,
|
||||
Agent,
|
||||
defines,
|
||||
logger
|
||||
)
|
||||
|
||||
from tools import (
|
||||
DateTime,
|
||||
WeatherForecast,
|
||||
TickerValue,
|
||||
tools
|
||||
logger,
|
||||
)
|
||||
|
||||
CONTEXT_VERSION=2
|
||||
@ -70,11 +64,17 @@ rags = [
|
||||
]
|
||||
|
||||
system_message = f"""
|
||||
Launched on {DateTime()}.
|
||||
Launched on {Tools.DateTime()}.
|
||||
|
||||
You have access to tools to get real time access to:
|
||||
- AnalyzeSite: Allows you to look up information on the Internet
|
||||
- TickerValue: Allows you to find stock price values
|
||||
- DateTime: Allows you to get the current date and time
|
||||
- WeatherForecast: Allows you to get the weather forecast for a given location
|
||||
|
||||
When answering queries, follow these steps:
|
||||
|
||||
- First analyze the query to determine if real-time information might be helpful
|
||||
- First analyze the query to determine if real-time information from the tools might be helpful
|
||||
- Even when <|context|> is provided, consider whether the tools would provide more current or comprehensive information
|
||||
- Use the provided tools whenever they would enhance your response, regardless of whether context is also available
|
||||
- When presenting weather forecasts, include relevant emojis immediately before the corresponding text. For example, for a sunny day, say \"☀️ Sunny\" or if the forecast says there will be \"rain showers, say \"🌧️ Rain showers\". Use this mapping for weather emojis: Sunny: ☀️, Cloudy: ☁️, Rainy: 🌧️, Snowy: ❄️
|
||||
@ -88,7 +88,7 @@ Always use tools and <|context|> when possible. Be concise, and never make up in
|
||||
"""
|
||||
|
||||
system_generate_resume = f"""
|
||||
Launched on {DateTime()}.
|
||||
Launched on {Tools.DateTime()}.
|
||||
|
||||
You are a professional resume writer. Your task is to write a concise, polished, and tailored resume for a specific job based only on the individual's <|context|>.
|
||||
|
||||
@ -119,7 +119,7 @@ Structure the resume professionally with the following sections where applicable
|
||||
""".strip()
|
||||
|
||||
system_fact_check = f"""
|
||||
Launched on {DateTime()}.
|
||||
Launched on {Tools.DateTime()}.
|
||||
|
||||
You are a professional resume fact checker. Your task is to identify any inaccuracies in the <|resume|> based on the individual's <|context|>.
|
||||
|
||||
@ -133,7 +133,7 @@ When answering queries, follow these steps:
|
||||
""".strip()
|
||||
|
||||
system_fact_check_QA = f"""
|
||||
Launched on {DateTime()}.
|
||||
Launched on {Tools.DateTime()}.
|
||||
|
||||
You are a professional resume fact checker.
|
||||
|
||||
@ -143,7 +143,7 @@ Your task is to answer questions about the <|fact_check|> you generated based on
|
||||
"""
|
||||
|
||||
system_job_description = f"""
|
||||
Launched on {DateTime()}.
|
||||
Launched on {Tools.DateTime()}.
|
||||
|
||||
You are a hiring and job placing specialist. Your task is to answers about a job description.
|
||||
|
||||
@ -261,66 +261,6 @@ def parse_args():
|
||||
|
||||
# %%
|
||||
|
||||
async def AnalyzeSite(llm, model: str, url : str, question : str):
|
||||
"""
|
||||
Fetches content from a URL, extracts the text, and uses Ollama to summarize it.
|
||||
|
||||
Args:
|
||||
url (str): The URL of the website to summarize
|
||||
|
||||
Returns:
|
||||
str: A summary of the website content
|
||||
"""
|
||||
try:
|
||||
# Fetch the webpage
|
||||
headers = {
|
||||
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36"
|
||||
}
|
||||
logger.info(f"Fetching {url}")
|
||||
response = requests.get(url, headers=headers, timeout=10)
|
||||
response.raise_for_status()
|
||||
logger.info(f"{url} returned. Processing...")
|
||||
# Parse the HTML
|
||||
soup = BeautifulSoup(response.text, "html.parser")
|
||||
|
||||
# Remove script and style elements
|
||||
for script in soup(["script", "style"]):
|
||||
script.extract()
|
||||
|
||||
# Get text content
|
||||
text = soup.get_text(separator=" ", strip=True)
|
||||
|
||||
# Clean up text (remove extra whitespace)
|
||||
lines = (line.strip() for line in text.splitlines())
|
||||
chunks = (phrase.strip() for line in lines for phrase in line.split(" "))
|
||||
text = " ".join(chunk for chunk in chunks if chunk)
|
||||
|
||||
# Limit text length if needed (Ollama may have token limits)
|
||||
max_chars = 100000
|
||||
if len(text) > max_chars:
|
||||
text = text[:max_chars] + "..."
|
||||
|
||||
# Create Ollama client
|
||||
# logger.info(f"Requesting summary of: {text}")
|
||||
|
||||
# Generate summary using Ollama
|
||||
prompt = f"CONTENTS:\n\n{text}\n\n{question}"
|
||||
response = llm.generate(model=model,
|
||||
system="You are given the contents of {url}. Answer the question about the contents",
|
||||
prompt=prompt)
|
||||
|
||||
#logger.info(response["response"])
|
||||
|
||||
return {
|
||||
"source": "summarizer-llm",
|
||||
"content": response["response"],
|
||||
"metadata": DateTime()
|
||||
}
|
||||
|
||||
except requests.exceptions.RequestException as e:
|
||||
return f"Error fetching the URL: {str(e)}"
|
||||
except Exception as e:
|
||||
return f"Error processing the website content: {str(e)}"
|
||||
|
||||
# %%
|
||||
|
||||
@ -332,14 +272,7 @@ def is_valid_uuid(value):
|
||||
except (ValueError, TypeError):
|
||||
return False
|
||||
|
||||
def default_tools(tools: list[dict[str, Any]]) -> list[dict[str, Any]]:
|
||||
return [{**tool, "enabled": True} for tool in tools]
|
||||
|
||||
def find_summarize_tool(tools):
|
||||
return [{**tool, "enabled": True} for tool in tools if tool.get("name", "") == "AnalyzeSite"]
|
||||
|
||||
def llm_tools(tools):
|
||||
return [tool for tool in tools if tool.get("enabled", False) == True]
|
||||
|
||||
|
||||
|
||||
@ -514,7 +447,7 @@ class WebServer:
|
||||
response["rags"] = context.rags
|
||||
case "tools":
|
||||
logger.info(f"Resetting {reset_operation}")
|
||||
context.tools = default_tools(tools)
|
||||
context.tools = Tools.default_tools(Tools.tools)
|
||||
response["tools"] = context.tools
|
||||
case "history":
|
||||
reset_map = {
|
||||
@ -676,9 +609,15 @@ class WebServer:
|
||||
|
||||
@self.app.post("/api/context")
|
||||
async def create_context():
|
||||
context = self.create_context()
|
||||
logger.info(f"Generated new agent as {context.id}")
|
||||
return JSONResponse({ "id": context.id })
|
||||
try:
|
||||
context = self.create_context()
|
||||
logger.info(f"Generated new agent as {context.id}")
|
||||
return JSONResponse({ "id": context.id })
|
||||
except Exception as e:
|
||||
logger.error(f"get_history error: {str(e)}")
|
||||
import traceback
|
||||
logger.error(traceback.format_exc())
|
||||
return JSONResponse({"error": str(e)}, status_code=404)
|
||||
|
||||
@self.app.get("/api/history/{context_id}/{agent_type}")
|
||||
async def get_history(context_id: str, agent_type: str, request: Request):
|
||||
@ -829,7 +768,8 @@ class WebServer:
|
||||
"""
|
||||
if not self.file_watcher:
|
||||
raise Exception("File watcher not initialized")
|
||||
|
||||
if not context_id:
|
||||
context_id = str(uuid4())
|
||||
logger.info(f"Creating new context with ID: {context_id}")
|
||||
context = Context(id=context_id, file_watcher=self.file_watcher)
|
||||
|
||||
@ -841,7 +781,7 @@ class WebServer:
|
||||
# context.add_agent(Resume(system_prompt = system_generate_resume))
|
||||
# context.add_agent(JobDescription(system_prompt = system_job_description))
|
||||
# context.add_agent(FactCheck(system_prompt = system_fact_check))
|
||||
context.tools = default_tools(tools)
|
||||
context.tools = Tools.default_tools(Tools.tools)
|
||||
context.rags = rags.copy()
|
||||
|
||||
logger.info(f"{context.id} created and added to contexts.")
|
||||
@ -854,67 +794,67 @@ class WebServer:
|
||||
return max(defines.max_context, min(2048, ctx + ctx_buffer))
|
||||
|
||||
# %%
|
||||
async def handle_tool_calls(self, message):
|
||||
"""
|
||||
Process tool calls and yield status updates along the way.
|
||||
The last yielded item will be a tuple containing (tool_result, tools_used).
|
||||
"""
|
||||
tools_used = []
|
||||
all_responses = []
|
||||
# async def handle_tool_calls(self, message):
|
||||
# """
|
||||
# Process tool calls and yield status updates along the way.
|
||||
# The last yielded item will be a tuple containing (tool_result, tools_used).
|
||||
# """
|
||||
# tools_used = []
|
||||
# all_responses = []
|
||||
|
||||
for i, tool_call in enumerate(message["tool_calls"]):
|
||||
arguments = tool_call["function"]["arguments"]
|
||||
tool = tool_call["function"]["name"]
|
||||
# for i, tool_call in enumerate(message["tool_calls"]):
|
||||
# arguments = tool_call["function"]["arguments"]
|
||||
# tool = tool_call["function"]["name"]
|
||||
|
||||
# Yield status update before processing each tool
|
||||
yield {"status": "processing", "message": f"Processing tool {i+1}/{len(message['tool_calls'])}: {tool}..."}
|
||||
# # Yield status update before processing each tool
|
||||
# yield {"status": "processing", "message": f"Processing tool {i+1}/{len(message['tool_calls'])}: {tool}..."}
|
||||
|
||||
# Process the tool based on its type
|
||||
match tool:
|
||||
case "TickerValue":
|
||||
ticker = arguments.get("ticker")
|
||||
if not ticker:
|
||||
ret = None
|
||||
else:
|
||||
ret = TickerValue(ticker)
|
||||
tools_used.append({ "tool": f"{tool}({ticker})", "result": ret})
|
||||
# # Process the tool based on its type
|
||||
# match tool:
|
||||
# case "TickerValue":
|
||||
# ticker = arguments.get("ticker")
|
||||
# if not ticker:
|
||||
# ret = None
|
||||
# else:
|
||||
# ret = Tools.TickerValue(ticker)
|
||||
# tools_used.append({ "tool": f"{tool}({ticker})", "result": ret})
|
||||
|
||||
case "AnalyzeSite":
|
||||
url = arguments.get("url")
|
||||
question = arguments.get("question", "what is the summary of this content?")
|
||||
# case "AnalyzeSite":
|
||||
# url = arguments.get("url")
|
||||
# question = arguments.get("question", "what is the summary of this content?")
|
||||
|
||||
# Additional status update for long-running operations
|
||||
yield {"status": "processing", "message": f"Retrieving and summarizing content from {url}..."}
|
||||
ret = await AnalyzeSite(llm=self.llm, model=self.model, url=url, question=question)
|
||||
tools_used.append({ "tool": f"{tool}('{url}', '{question}')", "result": ret })
|
||||
# # Additional status update for long-running operations
|
||||
# yield {"status": "processing", "message": f"Retrieving and summarizing content from {url}..."}
|
||||
# ret = await Tools.AnalyzeSite(llm=self.llm, model=self.model, url=url, question=question)
|
||||
# tools_used.append({ "tool": f"{tool}('{url}', '{question}')", "result": ret })
|
||||
|
||||
case "DateTime":
|
||||
tz = arguments.get("timezone")
|
||||
ret = DateTime(tz)
|
||||
tools_used.append({ "tool": f"{tool}('{tz}')", "result": ret })
|
||||
# case "DateTime":
|
||||
# tz = arguments.get("timezone")
|
||||
# ret = Tools.DateTime(tz)
|
||||
# tools_used.append({ "tool": f"{tool}('{tz}')", "result": ret })
|
||||
|
||||
case "WeatherForecast":
|
||||
city = arguments.get("city")
|
||||
state = arguments.get("state")
|
||||
# case "WeatherForecast":
|
||||
# city = arguments.get("city")
|
||||
# state = arguments.get("state")
|
||||
|
||||
yield {"status": "processing", "message": f"Fetching weather data for {city}, {state}..."}
|
||||
ret = WeatherForecast(city, state)
|
||||
tools_used.append({ "tool": f"{tool}('{city}', '{state}')", "result": ret })
|
||||
# yield {"status": "processing", "message": f"Fetching weather data for {city}, {state}..."}
|
||||
# ret = Tools.WeatherForecast(city, state)
|
||||
# tools_used.append({ "tool": f"{tool}('{city}', '{state}')", "result": ret })
|
||||
|
||||
case _:
|
||||
ret = None
|
||||
# case _:
|
||||
# ret = None
|
||||
|
||||
# Build response for this tool
|
||||
tool_response = {
|
||||
"role": "tool",
|
||||
"content": str(ret),
|
||||
"name": tool_call["function"]["name"]
|
||||
}
|
||||
all_responses.append(tool_response)
|
||||
# # Build response for this tool
|
||||
# tool_response = {
|
||||
# "role": "tool",
|
||||
# "content": str(ret),
|
||||
# "name": tool_call["function"]["name"]
|
||||
# }
|
||||
# all_responses.append(tool_response)
|
||||
|
||||
# Yield the final result as the last item
|
||||
final_result = all_responses[0] if len(all_responses) == 1 else all_responses
|
||||
yield (final_result, tools_used)
|
||||
# # Yield the final result as the last item
|
||||
# final_result = all_responses[0] if len(all_responses) == 1 else all_responses
|
||||
# yield (final_result, tools_used)
|
||||
|
||||
def upsert_context(self, context_id = None) -> Context:
|
||||
"""
|
||||
@ -980,7 +920,7 @@ class WebServer:
|
||||
raise Exception("File watcher not initialized")
|
||||
|
||||
agent_type = agent.get_agent_type()
|
||||
logger.info(f"generate_response: {agent_type}")
|
||||
logger.info(f"generate_response: type - {agent_type} prompt - {content}")
|
||||
if agent_type == "chat":
|
||||
message = Message(prompt=content)
|
||||
async for message in agent.prepare_message(message):
|
||||
@ -991,22 +931,13 @@ class WebServer:
|
||||
if message.status != "done":
|
||||
yield message
|
||||
async for message in agent.process_message(self.llm, self.model, message):
|
||||
# logger.info(f"{agent_type}.process_message: {value.status} - {value.response}")
|
||||
if message.status == "error":
|
||||
yield message
|
||||
return
|
||||
if message.status != "done":
|
||||
yield message
|
||||
# async for value in agent.generate_llm_response(message):
|
||||
# logger.info(f"{agent_type}.generate_llm_response: {value.status} - {value.response}")
|
||||
# if value.status != "done":
|
||||
# yield value
|
||||
# if value.status == "error":
|
||||
# message.status = "error"
|
||||
# message.response = value.response
|
||||
# yield message
|
||||
# return
|
||||
logger.info("TODO: There is more to do...")
|
||||
logger.info(f"{agent_type}.process_message: {message.status} {f'...{message.response[-20:]}' if len(message.response) > 20 else message.response}")
|
||||
message.status = "done"
|
||||
yield message
|
||||
return
|
||||
|
||||
|
||||
366
src/tools.py
366
src/tools.py
@ -1,366 +0,0 @@
|
||||
# %%
|
||||
# Imports [standard]
|
||||
# Standard library modules (no try-except needed)
|
||||
import argparse
|
||||
import asyncio
|
||||
import anyio
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import queue
|
||||
import re
|
||||
import time
|
||||
from datetime import datetime
|
||||
|
||||
def try_import(module_name, pip_name=None):
|
||||
try:
|
||||
__import__(module_name)
|
||||
except ImportError:
|
||||
print(f"Module '{module_name}' not found. Install it using:")
|
||||
print(f" pip install {pip_name or module_name}")
|
||||
|
||||
# Third-party modules with import checks
|
||||
try_import('gradio')
|
||||
try_import('ollama')
|
||||
try_import('openai')
|
||||
try_import('pytz')
|
||||
try_import('requests')
|
||||
try_import('yfinance', 'yfinance')
|
||||
try_import('dotenv', 'python-dotenv')
|
||||
try_import('geopy', 'geopy')
|
||||
|
||||
from dotenv import load_dotenv
|
||||
from geopy.geocoders import Nominatim
|
||||
import gradio as gr
|
||||
import pytz
|
||||
import requests
|
||||
import yfinance as yf
|
||||
|
||||
# %%
|
||||
def WeatherForecast(city, state, country="USA"):
|
||||
"""
|
||||
Get weather information from weather.gov based on city, state, and country.
|
||||
|
||||
Args:
|
||||
city (str): City name
|
||||
state (str): State name or abbreviation
|
||||
country (str): Country name (defaults to "USA" as weather.gov is for US locations)
|
||||
|
||||
Returns:
|
||||
dict: Weather forecast information
|
||||
"""
|
||||
# Step 1: Get coordinates for the location using geocoding
|
||||
location = f"{city}, {state}, {country}"
|
||||
coordinates = get_coordinates(location)
|
||||
|
||||
if not coordinates:
|
||||
return {"error": f"Could not find coordinates for {location}"}
|
||||
|
||||
# Step 2: Get the forecast grid endpoint for the coordinates
|
||||
grid_endpoint = get_grid_endpoint(coordinates)
|
||||
|
||||
if not grid_endpoint:
|
||||
return {"error": f"Could not find weather grid for coordinates {coordinates}"}
|
||||
|
||||
# Step 3: Get the forecast data from the grid endpoint
|
||||
forecast = get_forecast(grid_endpoint)
|
||||
|
||||
if not forecast['location']:
|
||||
forecast['location'] = location
|
||||
|
||||
return forecast
|
||||
|
||||
def get_coordinates(location):
|
||||
"""Convert a location string to latitude and longitude using Nominatim geocoder."""
|
||||
try:
|
||||
# Create a geocoder with a meaningful user agent
|
||||
geolocator = Nominatim(user_agent="weather_app_example")
|
||||
|
||||
# Get the location
|
||||
location_data = geolocator.geocode(location)
|
||||
|
||||
if location_data:
|
||||
return {
|
||||
"latitude": location_data.latitude,
|
||||
"longitude": location_data.longitude
|
||||
}
|
||||
else:
|
||||
print(f"Location not found: {location}")
|
||||
return None
|
||||
except Exception as e:
|
||||
print(f"Error getting coordinates: {e}")
|
||||
return None
|
||||
|
||||
def get_grid_endpoint(coordinates):
|
||||
"""Get the grid endpoint from weather.gov based on coordinates."""
|
||||
try:
|
||||
lat = coordinates["latitude"]
|
||||
lon = coordinates["longitude"]
|
||||
|
||||
# Define headers for the API request
|
||||
headers = {
|
||||
"User-Agent": "WeatherAppExample/1.0 (your_email@example.com)",
|
||||
"Accept": "application/geo+json"
|
||||
}
|
||||
|
||||
# Make the request to get the grid endpoint
|
||||
url = f"https://api.weather.gov/points/{lat},{lon}"
|
||||
response = requests.get(url, headers=headers)
|
||||
|
||||
if response.status_code == 200:
|
||||
data = response.json()
|
||||
return data["properties"]["forecast"]
|
||||
else:
|
||||
print(f"Error getting grid: {response.status_code} - {response.text}")
|
||||
return None
|
||||
except Exception as e:
|
||||
print(f"Error in get_grid_endpoint: {e}")
|
||||
return None
|
||||
|
||||
# Weather related function
|
||||
|
||||
def get_forecast(grid_endpoint):
|
||||
"""Get the forecast data from the grid endpoint."""
|
||||
try:
|
||||
# Define headers for the API request
|
||||
headers = {
|
||||
"User-Agent": "WeatherAppExample/1.0 (your_email@example.com)",
|
||||
"Accept": "application/geo+json"
|
||||
}
|
||||
|
||||
# Make the request to get the forecast
|
||||
response = requests.get(grid_endpoint, headers=headers)
|
||||
|
||||
if response.status_code == 200:
|
||||
data = response.json()
|
||||
|
||||
# Extract the relevant forecast information
|
||||
periods = data["properties"]["periods"]
|
||||
|
||||
# Process the forecast data into a simpler format
|
||||
forecast = {
|
||||
"location": data["properties"].get("relativeLocation", {}).get("properties", {}),
|
||||
"updated": data["properties"].get("updated", ""),
|
||||
"periods": []
|
||||
}
|
||||
|
||||
for period in periods:
|
||||
forecast["periods"].append({
|
||||
"name": period.get("name", ""),
|
||||
"temperature": period.get("temperature", ""),
|
||||
"temperatureUnit": period.get("temperatureUnit", ""),
|
||||
"windSpeed": period.get("windSpeed", ""),
|
||||
"windDirection": period.get("windDirection", ""),
|
||||
"shortForecast": period.get("shortForecast", ""),
|
||||
"detailedForecast": period.get("detailedForecast", "")
|
||||
})
|
||||
|
||||
return forecast
|
||||
else:
|
||||
print(f"Error getting forecast: {response.status_code} - {response.text}")
|
||||
return {"error": f"API Error: {response.status_code}"}
|
||||
except Exception as e:
|
||||
print(f"Error in get_forecast: {e}")
|
||||
return {"error": f"Exception: {str(e)}"}
|
||||
|
||||
# Example usage
|
||||
def do_weather():
|
||||
city = input("Enter city: ")
|
||||
state = input("Enter state: ")
|
||||
country = input("Enter country (default USA): ") or "USA"
|
||||
|
||||
print(f"Getting weather for {city}, {state}, {country}...")
|
||||
weather_data = WeatherForecast(city, state, country)
|
||||
|
||||
if "error" in weather_data:
|
||||
print(f"Error: {weather_data['error']}")
|
||||
else:
|
||||
print("\nWeather Forecast:")
|
||||
print(f"Location: {weather_data.get('location', {}).get('city', city)}, {weather_data.get('location', {}).get('state', state)}")
|
||||
print(f"Last Updated: {weather_data.get('updated', 'N/A')}")
|
||||
print("\nForecast Periods:")
|
||||
|
||||
for period in weather_data.get("periods", []):
|
||||
print(f"\n{period['name']}:")
|
||||
print(f" Temperature: {period['temperature']}{period['temperatureUnit']}")
|
||||
print(f" Wind: {period['windSpeed']} {period['windDirection']}")
|
||||
print(f" Forecast: {period['shortForecast']}")
|
||||
print(f" Details: {period['detailedForecast']}")
|
||||
|
||||
# %%
|
||||
|
||||
# Stock related function
|
||||
def TickerValue(ticker_symbols):
|
||||
"""
|
||||
Look up the current price of a stock using its ticker symbol.
|
||||
|
||||
Args:
|
||||
ticker_symbol (str): The stock ticker symbol (e.g., 'AAPL' for Apple)
|
||||
|
||||
Returns:
|
||||
dict: Current stock information including price
|
||||
"""
|
||||
results = []
|
||||
print(f"TickerValue('{ticker_symbols}')")
|
||||
for ticker_symbol in ticker_symbols.split(','):
|
||||
ticker_symbol = ticker_symbol.strip()
|
||||
if ticker_symbol == "":
|
||||
continue
|
||||
# Create a Ticker object
|
||||
try:
|
||||
ticker = yf.Ticker(ticker_symbol)
|
||||
print(ticker)
|
||||
# Get the latest market data
|
||||
ticker_data = ticker.history(period="1d")
|
||||
|
||||
if ticker_data.empty:
|
||||
results.append({"error": f"No data found for ticker {ticker_symbol}"})
|
||||
continue
|
||||
|
||||
# Get the latest closing price
|
||||
latest_price = ticker_data['Close'].iloc[-1]
|
||||
|
||||
# Get some additional info
|
||||
info = ticker.info
|
||||
results.append({ 'symbol': ticker_symbol, 'price': latest_price })
|
||||
|
||||
except Exception as e:
|
||||
results.append({"error": f"Error fetching data for {ticker_symbol}: {str(e)}"})
|
||||
|
||||
return results[0] if len(results) == 1 else results
|
||||
#{
|
||||
# "symbol": ticker_symbol,
|
||||
# "price": latest_price,
|
||||
# "currency": info.get("currency", "Unknown"),
|
||||
# "company_name": info.get("shortName", "Unknown"),
|
||||
# "previous_close": info.get("previousClose", "Unknown"),
|
||||
# "market_cap": info.get("marketCap", "Unknown"),
|
||||
#}
|
||||
|
||||
|
||||
# %%
|
||||
def DateTime(timezone="America/Los_Angeles"):
|
||||
"""
|
||||
Returns the current date and time in the specified timezone in ISO 8601 format.
|
||||
|
||||
Args:
|
||||
timezone (str): Timezone name (e.g., "UTC", "America/New_York", "Europe/London")
|
||||
Default is "America/Los_Angeles"
|
||||
|
||||
Returns:
|
||||
str: Current date and time with timezone in the format YYYY-MM-DDTHH:MM:SS+HH:MM
|
||||
"""
|
||||
try:
|
||||
if timezone == 'system' or timezone == '' or not timezone:
|
||||
timezone = 'America/Los_Angeles'
|
||||
# Get current UTC time (timezone-aware)
|
||||
local_tz = pytz.timezone("America/Los_Angeles")
|
||||
local_now = datetime.now(tz=local_tz)
|
||||
|
||||
# Convert to target timezone
|
||||
target_tz = pytz.timezone(timezone)
|
||||
target_time = local_now.astimezone(target_tz)
|
||||
|
||||
return target_time.isoformat()
|
||||
except Exception as e:
|
||||
return {'error': f"Invalid timezone {timezone}: {str(e)}"}
|
||||
|
||||
|
||||
# %%
|
||||
tools = [ {
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "TickerValue",
|
||||
"description": "Get the current stock price of one or more ticker symbols. Returns an array of objects with 'symbol' and 'price' fields. Call this whenever you need to know the latest value of stock ticker symbols, for example when a user asks 'How much is Intel trading at?' or 'What are the prices of AAPL and MSFT?'",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"ticker": {
|
||||
"type": "string",
|
||||
"description": "The company stock ticker symbol. For multiple tickers, provide a comma-separated list (e.g., 'AAPL,MSFT,GOOGL').",
|
||||
},
|
||||
},
|
||||
"required": ["ticker"],
|
||||
"additionalProperties": False
|
||||
}
|
||||
}
|
||||
}, {
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "AnalyzeSite",
|
||||
"description": "Downloads the requested site and asks a second LLM agent to answer the question based on the site content. For example if the user says 'What are the top headlines on cnn.com?' you would use AnalyzeSite to get the answer. Only use this if the user asks about a specific site or company.",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"url": {
|
||||
"type": "string",
|
||||
"description": "The website URL to download and process",
|
||||
},
|
||||
"question": {
|
||||
"type": "string",
|
||||
"description": "The question to ask the second LLM about the content",
|
||||
},
|
||||
},
|
||||
"required": ["url", "question"],
|
||||
"additionalProperties": False
|
||||
},
|
||||
"returns": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"source": {
|
||||
"type": "string",
|
||||
"description": "Identifier for the source LLM"
|
||||
},
|
||||
"content": {
|
||||
"type": "string",
|
||||
"description": "The complete response from the second LLM"
|
||||
},
|
||||
"metadata": {
|
||||
"type": "object",
|
||||
"description": "Additional information about the response"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}, {
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "DateTime",
|
||||
"description": "Get the current date and time in a specified timezone. For example if a user asks 'What time is it in Poland?' you would pass the Warsaw timezone to DateTime.",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"timezone": {
|
||||
"type": "string",
|
||||
"description": "Timezone name (e.g., 'UTC', 'America/New_York', 'Europe/London', 'America/Los_Angeles'). Default is 'America/Los_Angeles'."
|
||||
}
|
||||
},
|
||||
"required": []
|
||||
}
|
||||
}
|
||||
}, {
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "WeatherForecast",
|
||||
"description": "Get the full weather forecast as structured data for a given CITY and STATE location in the United States. For example, if the user asks 'What is the weather in Portland?' or 'What is the forecast for tomorrow?' use the provided data to answer the question.",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"city": {
|
||||
"type": "string",
|
||||
"description": "City to find the weather forecast (e.g., 'Portland', 'Seattle').",
|
||||
"minLength": 2
|
||||
},
|
||||
"state": {
|
||||
"type": "string",
|
||||
"description": "State to find the weather forecast (e.g., 'OR', 'WA').",
|
||||
"minLength": 2
|
||||
}
|
||||
},
|
||||
"required": [ "city", "state" ],
|
||||
"additionalProperties": False
|
||||
}
|
||||
}
|
||||
}]
|
||||
|
||||
__all__ = [ 'tools', 'DateTime', 'WeatherForecast', 'TickerValue' ]
|
||||
@ -1,4 +1,9 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Optional, Type
|
||||
import importlib
|
||||
from pydantic import BaseModel
|
||||
from typing import Type
|
||||
|
||||
from . import defines
|
||||
from . rag import ChromaDBFileWatcher, start_file_watcher
|
||||
@ -8,7 +13,7 @@ from . context import Context
|
||||
from . import agents
|
||||
from . setup_logging import setup_logging
|
||||
|
||||
from .agents import Agent, __all__ as agents_all
|
||||
from .agents import class_registry, AnyAgent, Agent, __all__ as agents_all
|
||||
|
||||
__all__ = [
|
||||
'Agent',
|
||||
@ -25,12 +30,7 @@ __all__ = [
|
||||
Agent.model_rebuild()
|
||||
Context.model_rebuild()
|
||||
|
||||
import importlib
|
||||
from pydantic import BaseModel
|
||||
from typing import Type
|
||||
|
||||
# Assuming class_registry is available from agents/__init__.py
|
||||
from .agents import class_registry, AnyAgent
|
||||
|
||||
logger = setup_logging(level=defines.logging_level)
|
||||
|
||||
|
||||
@ -46,5 +46,5 @@ for path in package_dir.glob("*.py"):
|
||||
continue
|
||||
except Exception as e:
|
||||
logger.error(f"Error processing {full_module_name}: {e}")
|
||||
continue
|
||||
raise e
|
||||
|
||||
|
||||
@ -56,6 +56,10 @@ class Agent(BaseModel, ABC):
|
||||
"""Return the set of valid agent_type values."""
|
||||
return set(get_args(cls.__annotations__["agent_type"]))
|
||||
|
||||
def agent_function_display(self):
|
||||
import inspect
|
||||
logger.info(f"{self.agent_type} - {inspect.stack()[1].function}")
|
||||
|
||||
def set_context(self, context):
|
||||
object.__setattr__(self, "context", context)
|
||||
|
||||
@ -63,195 +67,6 @@ class Agent(BaseModel, ABC):
|
||||
def get_agent_type(self):
|
||||
return self._agent_type
|
||||
|
||||
async def prepare_message(self, message:Message) -> AsyncGenerator[Message, None]:
|
||||
"""
|
||||
Prepare message with context information in message.preamble
|
||||
"""
|
||||
# Generate RAG content if enabled, based on the content
|
||||
rag_context = ""
|
||||
if not message.disable_rag:
|
||||
# Gather RAG results, yielding each result
|
||||
# as it becomes available
|
||||
for value in self.context.generate_rag_results(message):
|
||||
logger.info(f"RAG: {value.status} - {value.response}")
|
||||
if value.status != "done":
|
||||
yield value
|
||||
if value.status == "error":
|
||||
message.status = "error"
|
||||
message.response = value.response
|
||||
yield message
|
||||
return
|
||||
|
||||
if message.metadata["rag"]:
|
||||
for rag_collection in message.metadata["rag"]:
|
||||
for doc in rag_collection["documents"]:
|
||||
rag_context += f"{doc}\n"
|
||||
|
||||
if rag_context:
|
||||
message["context"] = rag_context
|
||||
|
||||
if self.context.user_resume:
|
||||
message["resume"] = self.content.user_resume
|
||||
|
||||
if message.preamble:
|
||||
preamble_types = [f"<|{p}|>" for p in message.preamble.keys()]
|
||||
preamble_types_AND = " and ".join(preamble_types)
|
||||
preamble_types_OR = " or ".join(preamble_types)
|
||||
message.preamble["rules"] = f"""\
|
||||
- Answer the question based on the information provided in the {preamble_types_AND} sections by incorporate it seamlessly and refer to it using natural language instead of mentioning {preamble_or_types} or quoting it directly.
|
||||
- If there is no information in these sections, answer based on your knowledge.
|
||||
- Avoid phrases like 'According to the {preamble_types[0]}' or similar references to the {preamble_types_OR}.
|
||||
"""
|
||||
message.preamble["question"] = "Use that information to respond to:"
|
||||
else:
|
||||
message.preamble["question"] = "Respond to:"
|
||||
|
||||
message.system_prompt = self.system_prompt
|
||||
message.status = "done"
|
||||
yield message
|
||||
return
|
||||
|
||||
async def generate_llm_response(self, message: Message) -> AsyncGenerator[Message, None]:
|
||||
if self.context.processing:
|
||||
logger.info("TODO: Implement delay queing; busy for same agent, otherwise return queue size and estimated wait time")
|
||||
message.status = "error"
|
||||
message.response = "Busy processing another request."
|
||||
yield message
|
||||
return
|
||||
|
||||
self.context.processing = True
|
||||
|
||||
messages = []
|
||||
|
||||
for value in self.llm.chat(
|
||||
model=self.model,
|
||||
messages=messages,
|
||||
#tools=llm_tools(context.tools) if message.enable_tools else None,
|
||||
options={ "num_ctx": message.ctx_size }
|
||||
):
|
||||
logger.info(f"LLM: {value.status} - {value.response}")
|
||||
if value.status != "done":
|
||||
message.status = value.status
|
||||
message.response = value.response
|
||||
yield message
|
||||
if value.status == "error":
|
||||
return
|
||||
response = value
|
||||
|
||||
message.metadata["eval_count"] += response["eval_count"]
|
||||
message.metadata["eval_duration"] += response["eval_duration"]
|
||||
message.metadata["prompt_eval_count"] += response["prompt_eval_count"]
|
||||
message.metadata["prompt_eval_duration"] += response["prompt_eval_duration"]
|
||||
agent.context_tokens = response["prompt_eval_count"] + response["eval_count"]
|
||||
|
||||
tools_used = []
|
||||
|
||||
yield {"status": "processing", "message": "Initial response received..."}
|
||||
|
||||
if "tool_calls" in response.get("message", {}):
|
||||
yield {"status": "processing", "message": "Processing tool calls..."}
|
||||
|
||||
tool_message = response["message"]
|
||||
tool_result = None
|
||||
|
||||
# Process all yielded items from the handler
|
||||
async for item in self.handle_tool_calls(tool_message):
|
||||
if isinstance(item, tuple) and len(item) == 2:
|
||||
# This is the final result tuple (tool_result, tools_used)
|
||||
tool_result, tools_used = item
|
||||
else:
|
||||
# This is a status update, forward it
|
||||
yield item
|
||||
|
||||
message_dict = {
|
||||
"role": tool_message.get("role", "assistant"),
|
||||
"content": tool_message.get("content", "")
|
||||
}
|
||||
|
||||
if "tool_calls" in tool_message:
|
||||
message_dict["tool_calls"] = [
|
||||
{"function": {"name": tc["function"]["name"], "arguments": tc["function"]["arguments"]}}
|
||||
for tc in tool_message["tool_calls"]
|
||||
]
|
||||
|
||||
pre_add_index = len(messages)
|
||||
messages.append(message_dict)
|
||||
|
||||
if isinstance(tool_result, list):
|
||||
messages.extend(tool_result)
|
||||
else:
|
||||
if tool_result:
|
||||
messages.append(tool_result)
|
||||
|
||||
message.metadata["tools"] = tools_used
|
||||
|
||||
# Estimate token length of new messages
|
||||
ctx_size = self.get_optimal_ctx_size(agent.context_tokens, messages=messages[pre_add_index:])
|
||||
yield {"status": "processing", "message": "Generating final response...", "num_ctx": ctx_size }
|
||||
# Decrease creativity when processing tool call requests
|
||||
response = self.llm.chat(model=self.model, messages=messages, stream=False, options={ "num_ctx": ctx_size }) #, "temperature": 0.5 })
|
||||
message.metadata["eval_count"] += response["eval_count"]
|
||||
message.metadata["eval_duration"] += response["eval_duration"]
|
||||
message.metadata["prompt_eval_count"] += response["prompt_eval_count"]
|
||||
message.metadata["prompt_eval_duration"] += response["prompt_eval_duration"]
|
||||
agent.context_tokens = response["prompt_eval_count"] + response["eval_count"]
|
||||
|
||||
reply = response["message"]["content"]
|
||||
message.response = reply
|
||||
message.metadata["origin"] = agent.agent_type
|
||||
# final_message = {"role": "assistant", "content": reply }
|
||||
|
||||
# # history is provided to the LLM and should not have additional metadata
|
||||
# llm_history.append(final_message)
|
||||
|
||||
# user_history is provided to the REST API and does not include CONTEXT
|
||||
# It does include metadata
|
||||
# final_message["metadata"] = message.metadata
|
||||
# user_history.append({**final_message, "origin": message.metadata["origin"]})
|
||||
|
||||
# Return the REST API with metadata
|
||||
yield {
|
||||
"status": "done",
|
||||
"message": {
|
||||
**message.model_dump(mode='json'),
|
||||
}
|
||||
}
|
||||
|
||||
self.context.processing = False
|
||||
return
|
||||
|
||||
async def process_message(self, llm: Any, model: str, message:Message) -> AsyncGenerator[Message, None]:
|
||||
message.full_content = ""
|
||||
for i, p in enumerate(message.preamble.keys()):
|
||||
message.full_content += '' if i == 0 else '\n\n' + f"<|{p}|>{message.preamble[p].strip()}\n"
|
||||
|
||||
# Estimate token length of new messages
|
||||
message.ctx_size = self.context.get_optimal_ctx_size(self.context_tokens, messages=message.full_content)
|
||||
|
||||
message.response = f"Processing {'RAG augmented ' if message.metadata['rag'] else ''}query..."
|
||||
message.status = "thinking"
|
||||
yield message
|
||||
|
||||
for value in self.generate_llm_response(message):
|
||||
logger.info(f"LLM: {value.status} - {value.response}")
|
||||
if value.status != "done":
|
||||
yield value
|
||||
if value.status == "error":
|
||||
return
|
||||
|
||||
def get_and_reset_content_seed(self):
|
||||
tmp = self._content_seed
|
||||
self._content_seed = ""
|
||||
return tmp
|
||||
|
||||
def set_content_seed(self, content: str) -> None:
|
||||
"""Set the content seed for the agent."""
|
||||
self._content_seed = content
|
||||
|
||||
def get_content_seed(self) -> str:
|
||||
"""Get the content seed for the agent."""
|
||||
return self._content_seed
|
||||
|
||||
# Register the base agent
|
||||
registry.register(Agent._agent_type, Agent)
|
||||
|
||||
|
||||
@ -5,10 +5,14 @@ from typing_extensions import Annotated
|
||||
from abc import ABC, abstractmethod
|
||||
from typing_extensions import Annotated
|
||||
import logging
|
||||
from .base import Agent, registry
|
||||
from . base import Agent, registry
|
||||
from .. conversation import Conversation
|
||||
from .. message import Message
|
||||
from .. import defines
|
||||
from .. import tools as Tools
|
||||
from ollama import ChatResponse
|
||||
import json
|
||||
import time
|
||||
|
||||
class Chat(Agent, ABC):
|
||||
"""
|
||||
@ -22,12 +26,13 @@ class Chat(Agent, ABC):
|
||||
"""
|
||||
Prepare message with context information in message.preamble
|
||||
"""
|
||||
self.agent_function_display()
|
||||
if not self.context:
|
||||
raise ValueError("Context is not set for this agent.")
|
||||
|
||||
# Generate RAG content if enabled, based on the content
|
||||
rag_context = ""
|
||||
if not message.disable_rag:
|
||||
if message.enable_rag:
|
||||
# Gather RAG results, yielding each result
|
||||
# as it becomes available
|
||||
for message in self.context.generate_rag_results(message):
|
||||
@ -69,20 +74,128 @@ class Chat(Agent, ABC):
|
||||
yield message
|
||||
return
|
||||
|
||||
async def generate_llm_response(self, llm: Any, model: str, message: Message) -> AsyncGenerator[Message, None]:
|
||||
async def process_tool_calls(self, llm: Any, model: str, message: Message, tool_message: Any, messages: List[Any]) -> AsyncGenerator[Message, None]:
|
||||
self.agent_function_display()
|
||||
if not self.context:
|
||||
raise ValueError("Context is not set for this agent.")
|
||||
raise ValueError("Context is not set for this agent.")
|
||||
if not message.metadata["tools"]:
|
||||
raise ValueError("tools field not initialized")
|
||||
logging.info(f"LLM - tool processing - {tool_message}")
|
||||
|
||||
if self.context.processing:
|
||||
logging.info("TODO: Implement delay queing; busy for same agent, otherwise return queue size and estimated wait time")
|
||||
message.status = "error"
|
||||
message.response = "Busy processing another request."
|
||||
tool_metadata = message.metadata["tools"]
|
||||
tool_metadata["messages"] = messages
|
||||
tool_metadata["tool_calls"] = []
|
||||
|
||||
message.status = "tooling"
|
||||
|
||||
for i, tool_call in enumerate(tool_message.tool_calls):
|
||||
arguments = tool_call.function.arguments
|
||||
tool = tool_call.function.name
|
||||
|
||||
# Yield status update before processing each tool
|
||||
message.response = f"Processing tool {i+1}/{len(tool_message.tool_calls)}: {tool}..."
|
||||
yield message
|
||||
|
||||
# Process the tool based on its type
|
||||
match tool:
|
||||
case "TickerValue":
|
||||
ticker = arguments.get("ticker")
|
||||
if not ticker:
|
||||
ret = None
|
||||
else:
|
||||
ret = Tools.TickerValue(ticker)
|
||||
|
||||
case "AnalyzeSite":
|
||||
url = arguments.get("url")
|
||||
question = arguments.get("question", "what is the summary of this content?")
|
||||
|
||||
# Additional status update for long-running operations
|
||||
message.response = f"Retrieving and summarizing content from {url}..."
|
||||
yield message
|
||||
ret = await Tools.AnalyzeSite(llm=llm, model=model, url=url, question=question)
|
||||
|
||||
case "DateTime":
|
||||
tz = arguments.get("timezone")
|
||||
ret = Tools.DateTime(tz)
|
||||
|
||||
case "WeatherForecast":
|
||||
city = arguments.get("city")
|
||||
state = arguments.get("state")
|
||||
|
||||
message.response = f"Fetching weather data for {city}, {state}..."
|
||||
yield message
|
||||
ret = Tools.WeatherForecast(city, state)
|
||||
|
||||
case _:
|
||||
ret = None
|
||||
|
||||
# Build response for this tool
|
||||
tool_response = {
|
||||
"role": "tool",
|
||||
"content": json.dumps(ret),
|
||||
"name": tool_call.function.name
|
||||
}
|
||||
|
||||
tool_metadata["tool_calls"].append(tool_response)
|
||||
|
||||
if len(tool_metadata["tool_calls"]) == 0:
|
||||
message.status = "done"
|
||||
yield message
|
||||
return
|
||||
|
||||
self.context.processing = True
|
||||
message_dict = {
|
||||
"role": tool_message.get("role", "assistant"),
|
||||
"content": tool_message.get("content", ""),
|
||||
"tool_calls": [
|
||||
{
|
||||
"function": {
|
||||
"name": tc["function"]["name"],
|
||||
"arguments": tc["function"]["arguments"]
|
||||
}
|
||||
}
|
||||
for tc in tool_message.tool_calls
|
||||
]
|
||||
}
|
||||
|
||||
self.conversation.add_message(message)
|
||||
messages.append(message_dict)
|
||||
messages.extend(tool_metadata["tool_calls"])
|
||||
|
||||
message.status = "thinking"
|
||||
|
||||
# Decrease creativity when processing tool call requests
|
||||
message.response = ""
|
||||
start_time = time.perf_counter()
|
||||
for response in llm.chat(
|
||||
model=model,
|
||||
messages=messages,
|
||||
stream=True,
|
||||
options={
|
||||
**message.metadata["options"],
|
||||
# "temperature": 0.5,
|
||||
}
|
||||
):
|
||||
# logging.info(f"LLM::Tools: {'done' if response.done else 'processing'} - {response.message}")
|
||||
message.status = "streaming"
|
||||
message.response += response.message.content
|
||||
if not response.done:
|
||||
yield message
|
||||
if response.done:
|
||||
message.metadata["eval_count"] += response.eval_count
|
||||
message.metadata["eval_duration"] += response.eval_duration
|
||||
message.metadata["prompt_eval_count"] += response.prompt_eval_count
|
||||
message.metadata["prompt_eval_duration"] += response.prompt_eval_duration
|
||||
self.context_tokens = response.prompt_eval_count + response.eval_count
|
||||
message.status = "done"
|
||||
end_time = time.perf_counter()
|
||||
message.metadata["timers"]["llm_with_tools"] = f"{(end_time - start_time):.4f}"
|
||||
message.status = "done"
|
||||
yield message
|
||||
return
|
||||
|
||||
async def generate_llm_response(self, llm: Any, model: str, message: Message) -> AsyncGenerator[Message, None]:
|
||||
self.agent_function_display()
|
||||
if not self.context:
|
||||
raise ValueError("Context is not set for this agent.")
|
||||
|
||||
messages = [
|
||||
item for m in self.conversation.messages
|
||||
@ -91,17 +204,120 @@ class Chat(Agent, ABC):
|
||||
{"role": "assistant", "content": m.response}
|
||||
]
|
||||
]
|
||||
messages.append({
|
||||
"role": "user",
|
||||
"content": message.full_content,
|
||||
})
|
||||
|
||||
message.status = "thinking"
|
||||
message.metadata["options"]={
|
||||
"seed": 8911,
|
||||
"num_ctx": message.metadata["ctx_size"] if message.metadata["ctx_size"] else defines.max_context,
|
||||
"temperature": 0.9, # Higher temperature to encourage tool usage
|
||||
}
|
||||
|
||||
message.metadata["timers"] = {}
|
||||
|
||||
use_tools = message.enable_tools and len(self.context.tools) > 0
|
||||
message.metadata["tools"] = {
|
||||
"available": Tools.llm_tools(self.context.tools),
|
||||
"used": False
|
||||
}
|
||||
|
||||
if use_tools:
|
||||
message.status = "thinking"
|
||||
message.response = f"Performing tool analysis step 1/2..."
|
||||
yield message
|
||||
|
||||
logging.info("Checking for LLM tool usage")
|
||||
start_time = time.perf_counter()
|
||||
# Tools are enabled and available, so query the LLM with a short token target to see if it will
|
||||
# use the tools
|
||||
response = llm.chat(
|
||||
model=model,
|
||||
messages=messages, #[{ "role": "system", "content": self.system_prompt}, {"role": "user", "content": message.prompt}],
|
||||
tools=message.metadata["tools"]["available"],
|
||||
options={
|
||||
**message.metadata["options"],
|
||||
#"num_predict": 1024, # "Low" token limit to cut off after tool call
|
||||
},
|
||||
stream=False # No need to stream the probe
|
||||
)
|
||||
end_time = time.perf_counter()
|
||||
message.metadata["timers"]["tool_check"] = f"{(end_time - start_time):.4f}"
|
||||
if not response.message.tool_calls:
|
||||
logging.info("LLM indicates tools will not be used")
|
||||
# The LLM will not use tools, so disable use_tools so we can stream the full response
|
||||
use_tools = False
|
||||
|
||||
if use_tools:
|
||||
logging.info("LLM indicates tools will be used")
|
||||
|
||||
# Tools are enabled and available and the LLM indicated it will use them
|
||||
message.metadata["tools"]["attempted"] = response.message.tool_calls
|
||||
message.response = f"Performing tool analysis step 2/2 (tool use suspected)..."
|
||||
yield message
|
||||
|
||||
logging.info(f"Performing LLM call with tools")
|
||||
start_time = time.perf_counter()
|
||||
response = llm.chat(
|
||||
model=model,
|
||||
messages=messages,
|
||||
tools=message.metadata["tools"]["available"],
|
||||
options={
|
||||
**message.metadata["options"],
|
||||
},
|
||||
stream=False
|
||||
)
|
||||
end_time = time.perf_counter()
|
||||
message.metadata["timers"]["non_streaming"] = f"{(end_time - start_time):.4f}"
|
||||
|
||||
if not response:
|
||||
message.status = "error"
|
||||
message.response = "No response from LLM."
|
||||
yield message
|
||||
return
|
||||
|
||||
if response.message.tool_calls:
|
||||
message.metadata["tools"]["used"] = response.message.tool_calls
|
||||
# Process all yielded items from the handler
|
||||
start_time = time.perf_counter()
|
||||
async for message in self.process_tool_calls(llm=llm, model=model, message=message, tool_message=response.message, messages=messages):
|
||||
if message.status == "error":
|
||||
yield message
|
||||
return
|
||||
yield message
|
||||
end_time = time.perf_counter()
|
||||
message.metadata["timers"]["process_tool_calls"] = f"{(end_time - start_time):.4f}"
|
||||
message.status = "done"
|
||||
return
|
||||
|
||||
logging.info("LLM indicated tools will be used, and then they weren't")
|
||||
message.response = response.message.content
|
||||
message.status = "done"
|
||||
yield message
|
||||
return
|
||||
|
||||
# not use_tools
|
||||
# Reset the response for streaming
|
||||
message.response = ""
|
||||
start_time = time.perf_counter()
|
||||
for response in llm.chat(
|
||||
model=model,
|
||||
messages=messages,
|
||||
#tools=llm_tools(context.tools) if message.enable_tools else None,
|
||||
options={ "num_ctx": message.metadata["ctx_size"] if message.metadata["ctx_size"] else defines.max_context },
|
||||
options={
|
||||
**message.metadata["options"],
|
||||
},
|
||||
stream=True,
|
||||
):
|
||||
if not response:
|
||||
message.status = "error"
|
||||
message.response = "No response from LLM."
|
||||
yield message
|
||||
return
|
||||
message.status = "streaming"
|
||||
message.response += response.message.content
|
||||
yield message
|
||||
if not response.done:
|
||||
yield message
|
||||
if response.done:
|
||||
message.metadata["eval_count"] += response.eval_count
|
||||
message.metadata["eval_duration"] += response.eval_duration
|
||||
@ -109,100 +325,28 @@ class Chat(Agent, ABC):
|
||||
message.metadata["prompt_eval_duration"] += response.prompt_eval_duration
|
||||
self.context_tokens = response.prompt_eval_count + response.eval_count
|
||||
message.status = "done"
|
||||
yield message
|
||||
|
||||
if not response:
|
||||
message.status = "error"
|
||||
message.response = "No response from LLM."
|
||||
yield message
|
||||
self.context.processing = False
|
||||
return
|
||||
|
||||
yield message
|
||||
self.context.processing = False
|
||||
return
|
||||
|
||||
tools_used = []
|
||||
|
||||
|
||||
if "tool_calls" in response.get("message", {}):
|
||||
message.status = "thinking"
|
||||
message.response = "Processing tool calls..."
|
||||
|
||||
tool_message = response["message"]
|
||||
tool_result = None
|
||||
|
||||
# Process all yielded items from the handler
|
||||
async for value in self.handle_tool_calls(tool_message):
|
||||
if isinstance(value, tuple) and len(value) == 2:
|
||||
# This is the final result tuple (tool_result, tools_used)
|
||||
tool_result, tools_used = value
|
||||
else:
|
||||
# This is a status update, forward it
|
||||
yield value
|
||||
|
||||
message_dict = {
|
||||
"role": tool_message.get("role", "assistant"),
|
||||
"content": tool_message.get("content", "")
|
||||
}
|
||||
|
||||
if "tool_calls" in tool_message:
|
||||
message_dict["tool_calls"] = [
|
||||
{"function": {"name": tc["function"]["name"], "arguments": tc["function"]["arguments"]}}
|
||||
for tc in tool_message["tool_calls"]
|
||||
]
|
||||
|
||||
pre_add_index = len(messages)
|
||||
messages.append(message_dict)
|
||||
|
||||
if isinstance(tool_result, list):
|
||||
messages.extend(tool_result)
|
||||
else:
|
||||
if tool_result:
|
||||
messages.append(tool_result)
|
||||
|
||||
message.metadata["tools"] = tools_used
|
||||
|
||||
# Estimate token length of new messages
|
||||
ctx_size = self.get_optimal_ctx_size(agent.context_tokens, messages=messages[pre_add_index:])
|
||||
yield {"status": "processing", "message": "Generating final response...", "num_ctx": ctx_size }
|
||||
# Decrease creativity when processing tool call requests
|
||||
response = self.llm.chat(model=self.model, messages=messages, stream=False, options={ "num_ctx": ctx_size }) #, "temperature": 0.5 })
|
||||
message.metadata["eval_count"] += response["eval_count"]
|
||||
message.metadata["eval_duration"] += response["eval_duration"]
|
||||
message.metadata["prompt_eval_count"] += response["prompt_eval_count"]
|
||||
message.metadata["prompt_eval_duration"] += response["prompt_eval_duration"]
|
||||
agent.context_tokens = response["prompt_eval_count"] + response["eval_count"]
|
||||
|
||||
reply = response["message"]["content"]
|
||||
message.response = reply
|
||||
message.metadata["origin"] = agent.agent_type
|
||||
# final_message = {"role": "assistant", "content": reply }
|
||||
|
||||
# # history is provided to the LLM and should not have additional metadata
|
||||
# llm_history.append(final_message)
|
||||
|
||||
# user_history is provided to the REST API and does not include CONTEXT
|
||||
# It does include metadata
|
||||
# final_message["metadata"] = message.metadata
|
||||
# user_history.append({**final_message, "origin": message.metadata["origin"]})
|
||||
|
||||
# Return the REST API with metadata
|
||||
yield {
|
||||
"status": "done",
|
||||
"message": {
|
||||
**message.model_dump(mode='json'),
|
||||
}
|
||||
}
|
||||
|
||||
self.context.processing = False
|
||||
end_time = time.perf_counter()
|
||||
message.metadata["timers"]["streamed"] = f"{(end_time - start_time):.4f}"
|
||||
return
|
||||
|
||||
async def process_message(self, llm: Any, model: str, message:Message) -> AsyncGenerator[Message, None]:
|
||||
self.agent_function_display()
|
||||
if not self.context:
|
||||
raise ValueError("Context is not set for this agent.")
|
||||
|
||||
message.full_content = f"<|system|>{self.system_prompt.strip()}\n"
|
||||
for i, p in enumerate(message.preamble.keys()):
|
||||
if self.context.processing:
|
||||
logging.info("TODO: Implement delay queing; busy for same agent, otherwise return queue size and estimated wait time")
|
||||
message.status = "error"
|
||||
message.response = "Busy processing another request."
|
||||
yield message
|
||||
return
|
||||
|
||||
self.context.processing = True
|
||||
|
||||
message.metadata["system_prompt"] = f"<|system|>{self.system_prompt.strip()}\n"
|
||||
for p in message.preamble.keys():
|
||||
message.full_content += f"\n<|{p}|>\n{message.preamble[p].strip()}\n"
|
||||
message.full_content += f"{message.prompt}"
|
||||
|
||||
@ -210,35 +354,25 @@ class Chat(Agent, ABC):
|
||||
message.metadata["ctx_size"] = self.context.get_optimal_ctx_size(self.context_tokens, messages=message.full_content)
|
||||
|
||||
message.response = f"Processing {'RAG augmented ' if message.metadata['rag'] else ''}query..."
|
||||
message.status = "searching"
|
||||
message.status = "thinking"
|
||||
yield message
|
||||
|
||||
async for message in self.generate_llm_response(llm, model, message):
|
||||
logging.info(f"LLM: {message.status} - {f'...{message.response[-20:]}' if len(message.response) > 20 else message.response}")
|
||||
# logging.info(f"LLM: {message.status} - {f'...{message.response[-20:]}' if len(message.response) > 20 else message.response}")
|
||||
if message.status == "error":
|
||||
yield message
|
||||
self.context.processing = False
|
||||
return
|
||||
if message.status != "done":
|
||||
yield message
|
||||
|
||||
yield message
|
||||
|
||||
# Done processing, add message to conversation
|
||||
message.status = "done"
|
||||
self.conversation.add_message(message)
|
||||
self.context.processing = False
|
||||
return
|
||||
|
||||
def get_and_reset_content_seed(self):
|
||||
tmp = self._content_seed
|
||||
self._content_seed = ""
|
||||
return tmp
|
||||
|
||||
def set_content_seed(self, content: str) -> None:
|
||||
"""Set the content seed for the agent."""
|
||||
self._content_seed = content
|
||||
|
||||
def get_content_seed(self) -> str:
|
||||
"""Get the content seed for the agent."""
|
||||
return self._content_seed
|
||||
|
||||
@classmethod
|
||||
def valid_agent_types(cls) -> set[str]:
|
||||
"""Return the set of valid agent_type values."""
|
||||
return set(get_args(cls.__annotations__["agent_type"]))
|
||||
|
||||
# Register the base agent
|
||||
registry.register(Chat._agent_type, Chat)
|
||||
|
||||
@ -11,7 +11,7 @@ import re
|
||||
from .message import Message
|
||||
from .rag import ChromaDBFileWatcher
|
||||
from . import defines
|
||||
|
||||
from . import tools as Tools
|
||||
from .agents import AnyAgent
|
||||
|
||||
logging.basicConfig(level=logging.INFO)
|
||||
@ -30,7 +30,7 @@ class Context(BaseModel):
|
||||
user_resume: Optional[str] = None
|
||||
user_job_description: Optional[str] = None
|
||||
user_facts: Optional[str] = None
|
||||
tools: List[dict] = []
|
||||
tools: List[dict] = Tools.default_tools(Tools.tools)
|
||||
rags: List[dict] = []
|
||||
message_history_length: int = 5
|
||||
context_tokens: int = 0
|
||||
@ -109,6 +109,7 @@ class Context(BaseModel):
|
||||
"umap_embedding_2d": umap_2d,
|
||||
"umap_embedding_3d": umap_3d
|
||||
})
|
||||
message.response = f"Results from {rag['name']} RAG: {len(chroma_results['documents'])} results."
|
||||
yield message
|
||||
|
||||
if entries == 0:
|
||||
|
||||
@ -7,8 +7,8 @@ class Message(BaseModel):
|
||||
prompt: str # Query to be answered
|
||||
|
||||
# Tunables
|
||||
disable_rag: bool = False
|
||||
disable_tools: bool = False
|
||||
enable_rag: bool = True
|
||||
enable_tools: bool = True
|
||||
|
||||
# Generated while processing message
|
||||
status: str = "" # Status of the message
|
||||
@ -18,7 +18,6 @@ class Message(BaseModel):
|
||||
response: str = "" # LLM response to the preamble + query
|
||||
metadata: dict[str, Any] = {
|
||||
"rag": List[dict[str, Any]],
|
||||
"tools": [],
|
||||
"eval_count": 0,
|
||||
"eval_duration": 0,
|
||||
"prompt_eval_count": 0,
|
||||
|
||||
@ -8,9 +8,10 @@ def setup_logging(level=defines.logging_level) -> logging.Logger:
|
||||
os.environ["TORCH_CPP_LOG_LEVEL"] = "ERROR"
|
||||
warnings.filterwarnings("ignore", message="Overriding a previously registered kernel")
|
||||
warnings.filterwarnings("ignore", message="Warning only once for all operators")
|
||||
warnings.filterwarnings("ignore", message="Couldn't find ffmpeg or avconv")
|
||||
warnings.filterwarnings("ignore", message=".*Couldn't find ffmpeg or avconv.*")
|
||||
warnings.filterwarnings("ignore", message="'force_all_finite' was renamed to")
|
||||
warnings.filterwarnings("ignore", message="n_jobs value 1 overridden")
|
||||
warnings.filterwarnings("ignore", message=".*websocket.*is deprecated")
|
||||
|
||||
numeric_level = getattr(logging, level.upper(), None)
|
||||
if not isinstance(numeric_level, int):
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user