DSC Conference Tutorials Day 1
In preparation for the DSC Conference we were offered 2 days of online webinar-style tutorials.
LLM Applications with LangChain
By Valira.ai
The first tutorial I decided to attend was on the topic of LLM applications with LangChain.
“The goal of this tutorial is to familiarize the participants with LangChain library, a Python framework for building applications based on large language models such as ChatGPT. We will introduce main LangChain concepts such as agents, chains and memory. To demonstrate the capabilities of LangChain, we will solve two problems that we might encounter in practice: a QA chatbot for answering questions over a collection of internal documents and an assistant for querying relational databases using natural language.”
What is LangChain ?
Similar to LlamaIndex and haystack
These LLMs are stateless (no awareness of context) and have no conversational ability (no memory)
LangChain offers prompt template to add pre-prompts to always give one type of answer for example. LangChain gives the ability to chain such components together.
First Use Case : Querying a database with natural language
Using data from kaggle.com and more precisely using the nba_api to get basketball data.
To understand the data, we need to know what data to pass it. So at first we give it a system prompt who tells it who it is, what it does and so on. We also tell it the structure of the database so that it knows what to expect.
We learned how to structure a query into a natural language query that is turned into an sql query by the LLM, which delivers a sql result and is finally transformed into a natural language answer.
This is done by stopping the query at the right time and using the result to create an answer to each step of the overall question.
One of its main problems is that it can only use a single query, there is no support yet for multiple chained queries. This problem should be solved in the future. It also fails in case the produced query is not valid since it will create an empty response and hallucinate a final response.
Another specific problem is that when querying using a first name, it will output an empty response since in this database names and first names are stored in the same variable. And since the produced query is looking for an exact match it will not work.
Second Use Case : QA Chatbot
QA Chatbots have several main problems :
- They’re not connected to the internet (don’t have latest data available)
- They’re trained with public data, not company data and would hallucinate answers if they don’t have data for a question.
- They’re expensive to train, so they need to be reusable for many different applications.
The training problem can be made easier by creating “Vector stores” that contain all company data that has been indexed and can be used to train the LLM for different applications within a company.
In this example, we used the NBA rulebook and a set of Wikipedia articles as training data.
To ensure the process works, it’s important to split the documentation into more manageable segments (because of input size limitations and to limit any excess “noise”).
Once the documentation has been split, it’s important to turn it into a vector store that contains all the documents but is ready to be used by the LLM.
It is then possible to use agents to string together multiple queries and handle them in succession to create complex queries.
Reimagining DataFrames: An Introduction to Graphs and GNNs with PyTorch Geometric
Graphs are networks of nodes and edges. They can be used to represent many different things, such as social networks, transportation networks, molecules, etc.
This second tutorial dove into some complex concepts that were too advanced for me but I gathered a few interesting topics to further explore in the future :
- pytorch (python library for machine learning)
- networkx (python library for network analysis and visualisation)
I will also explore their Google Collab files when I have more time.
Reinforcement learning through gamification
In this tutorial, participants will learn the basics of reinforcement learning by creating intelligent agents powered by artificial neural networks in a fun and interactive way. Participants will be guided towards creating a simple game world occupied by a player agent. Once an environment is set up and the concepts behind a game world are defined and explained participants will be instructed on how to create a population of intelligent agents that, through an iterative process, learn how to beat the created game efficiently. Complex concepts such as objective evaluation functions, neural networks will be covered interactively and visually.
by Luka Jovanovic Google Drive
Reinforcement learning is a method that allows us to use machine learning without being an expert in the field. It operates by deploying a model in a controlled environment where it can receive feedback, learn, and enhance its performance.
Gamification is a technique that transforms a task into a game by incorporating elements like scores, leaderboards, and reward systems. This approach is effective in promoting learning in humans and can be similarly applied to facilitate machine learning.
Various animals, including humans, engage in games as a safe way to experience and learn from life. This process often involves breaking down tasks into smaller ones and providing rewards upon successful completion. This same concept is utilized in machine learning through gamification.
The reward mechanism in machine learning involves agents interacting with an environment, observing it, and receiving rewards when they achieve success or make progress. We can assist the model in adapting to changes in the environment by adjusting its weights and biases.
In our specific case, we’re using a game similar to Flappy Bird. The model needs to consider four parameters to make a decision: the bird’s height, the distance to the next pipe, the height of the next pipe, and the bird’s velocity.
There are several steps we can take to begin:
- Create a population of 100 agents
- Allow them to play the game and record their scores
- Select the top-performing agents
- Combine these agents to create “offspring” (a process akin to natural selection)
- Introduce mutations in some offspring (random parameter changes that may potentially enhance the agent, similar to genetic mutations)
- Repeat the process
This method is known as the genetic algorithms.
Entity component systems provide a structure for game code, consisting of three parts:
Entities: These are objects in the game, like the bird or the pipes. Anything that holds data is an entity, and each entity should have a unique identifier for tracking purposes. Components: These are the properties of the entities, such as location or health status. They represent any information about an entity. Systems: These represent the game’s logic, dictating how entities interact with each other and move. Any logic not directly related to an entity falls under this category.
After discussing with Luka he suggested I explore a few interesting topics and algorithms. I’ve searched for and added a short description of each already, but I will need to explore them further in the future:
Metaheuristic Algorithms: Metaheuristic algorithms are high-level problem-independent algorithmic frameworks that provide a set of guidelines or strategies to develop heuristic optimization algorithms. They are designed to solve complex optimization problems where other traditional methods are inefficient. Examples include Genetic Algorithms, Simulated Annealing, and Tabu Search.
Sine-Cosine Algorithm (SCA): This is a population-based optimization algorithm, inspired by the mathematical sine and cosine functions. It’s used to find the least values of benchmarks functions. The algorithm uses sine and cosine functions to generate new solutions around the best solution found so far.
Ant Colony Optimization (ACO): ACO is a probabilistic technique used in problems that involve finding better paths through graphs. It’s inspired by the behavior of ants in finding paths from the colony to food. Ants lay down pheromones directing each other to resources while exploring their environment. The idea is to take the path where the concentration of pheromones is highest, leading to the discovery of the shortest routes.
Firefly Algorithm (FA): FA is a metaheuristic algorithm inspired by the flashing behavior of fireflies. The primary purpose of the firefly’s flash is to act as a signal system to attract other fireflies. In the Firefly Algorithm, attractiveness is associated with the brightness of the fireflies, and they move towards the ones that are more attractive or brighter. This behavior is used for solving optimization problems.
Artificial Bee Colony (ABC): ABC is an optimization algorithm based on the intelligent foraging behavior of honey bee swarm. It models the behavior of worker bees in finding the shortest route to flowers with the most nectar. In the context of optimization problems, the nectar amount is the quality (fitness) of solutions.
To improve the code he supplied, he suggested I look into premature convergence and how to improve it.f
He also mentioned the fact that it is possible to “save” the best birds and re-use them in a more complicated environment (moving pipes, other obstacles, etc…) or even a different one altogether.h
Another attendee also suggested i read the following article about Deep NeuroEvolution