Hosting GenAI

Comment

About the event

Run AI workloads on Linux and Kubernetes

Living Open Source Foundation is proud to present a unique event. On March 17th Sander van Vugt will be presenting contents from his new and upcoming video course "Hosting on-premises GenAI" live from the Living Open Source Foundation headquarters in Ibex Hill Lusaka. 

At the end of this session you will know how to host ChatGPT-alike functionality from within your own organization, and how to enhance it with your own custom data so that your internal documents are considered as well.

In this full day event the following topics will be covered:

1.    Understanding key components for hosting GenAI: LLMs and inference engines
2.    Hosting GenAI - even on low-end hardware without GPU
3.    Optimizing Linux for hosting GenAI inference. 
4.    Running a hardware friendly solution based on llama.cpp on Linux

Break

5.    Running a enterprise friendly solution based on vLLM on Linux
6.    Including your own documents using RAG
7.    Running a scalable AI solution on top of Kubernetes
8.    How all of this helps forward schools in rural Zambia

Note: Living Open Source Foundation sponsors Cheeba primary school in Mazabuka. This school doesn't have a library, which makes accessing knowledge for pupils difficult. By implementing this solution on low-end hardware, we will be able to provide pupils of the school access to a wealth of information.

Location on map

Get Directions