Herding Llama 3.1 with Elastic and LM Studio

Herding Llama 3.1 with Elastic and LM Studio | Elastic Blog

Skip to main content

New

Forrester Wave Leader, Q2 2025

Access report

About usPartnersSupport|ENLogin

[](http://www.elastic.co/)

• Elasticsearch

##### Elasticsearch for...

• ###### Context engineering Get the most relevant context to agents so that they deliver accurate and trusted outcomes
• ###### Vector database Efficiently create, store, and search vector embeddings
• ###### Search powered applications The speed, scale, and flexibility to power modern application experience
• ###### Logs Collect, search, explore, and act on large volumes
• ###### Threat protection Detect, investigate, and remediate cyber threats at scale on real-time data
• ###### Workflows Combine scripted automation with AI reasoning natively in Elasticsearch

##### Elasticsearch components

• ###### Elasticsearch A distributed, RESTful search and analytics engine
• ###### Kibana (Discover, Dashboards) Explore, visualize, and build dashboards using data stored in Elasticsearch
• ###### Elastic Agent Builder Build context-aware agents faster that incorporate all your data and deliver best-in-class relevance.
• ###### AutoOps Easy cluster management with performance recommendations, resource utilization, and cost insights
• ###### Piped query language Simplify workflows and accelerate query response for efficient data processing
• ###### Jina AI search models Jina AI is part of Elastic, bringing best-in-class models for embeddings, rerankers, and URL and doc extraction

##### Deployment options

• ###### Elastic Cloud Serverless Zero operational load so that you can build fasterStart free trial
• ###### Elastic Cloud Hosted Deploy and scale on any cloud in minutes with ultimate controlStart free trial
• ###### Self-managed Elasticsearch Run locally, via Kubernetes, or your own orchestrationDownload

• Solutions

##### Search

• ###### Ecommerce search Improve customers' search experience and drive conversion
• ###### Customer support search Help customers find support information quickly and easily
• ###### Search-driven apps Create engaging apps quickly and easily with Elasticsearch

Overview

##### Observability

• ###### Log analytics Centralize and analyze logs using Search AI to detect, investigate, and remediate incidents
• ###### Infrastructure monitoring Monitor, visualize, and analyze the health of your on-premises and cloud infrastructure
• ###### Digital experience monitoring Improve users' experience with real user monitoring (RUM), synthetic testing, and uptime monitoring
• ###### App performance monitoring Monitor, visualize, and analyze the performance and availability of your applications
• ###### AIOps Automatically detect, diagnose, and resolve issues faster with GenAl and ML
• ###### LLM observability Monitor and optimize LLM performance, cost, safety, and reliability

Overview

##### Security

• ###### Next-gen SIEM Detect, investigate, and respond to evolving threats with Al-driven security analytics
• ###### Workflows for security Automate alert triage, enrichment, and response natively. No separate SOAR required.
• ###### XDR and endpoint security Secure your endpoints, clouds, and containers with AI-driven insights
• ###### AI for security Automate your triage, investigation, and response workflows with Search AI

Overview

• Enterprise

##### Why Elastic?

• ###### 10x your data's value Learn how you can tap into your underutilized data with Search AI

Knowledge Hub

##### Industry

Financial servicesManufacturingPublic sectorRetailTelecommunicationsView all industries

##### Better together

• ###### Cloud providers Deploy with your favorite cloud marketplace: AWS, Azure, or Google Cloud
• ###### Elastic AI Ecosystem Use Elastic with built-in integrations with leading Al technology providers
• ###### Search AI Partner Program Partner with Elastic so we can find the answers, together

##### Accolades

• ###### AV-Comparatives Elastic earns Endpoint Prevention and Response Certification from AV-Comparatives
• ###### Forrester Wave™ Leader A Leader in The Forrester Wave™: Security Analytics Platforms, Q2 2025
• ###### Gartner Magic Quadrant Leader A Leader in 2025 Gartner® Magic Quadrant™ for Observability Platforms
• ###### IDC MarketScape Leader Leader in IDC MarketScape: Worldwide SIEM for Enterprise 2024

##### Customers

View all customers stories

[Search Docusign powers millions of e-signature searches daily with Elasticsearch](http://www.elastic.co/customers/docusign)

[Security UOL slashes incident resolution time by 80% with Elastic Security](http://www.elastic.co/customers/uol)

[Observability Pepsi boosts efficiency and reduces MTTR by 30% with Elastic Observability](http://www.elastic.co/customers/pepsico)

• Resources

##### Launch

• ###### Get started Follow along with beginner guides for each solution
• ###### Demo gallery Play in our hands-on sandbox and watch how-to videos
• ###### Downloads Download Elasticsearch now to get started for free
• ###### Integrations Easily connect Elasticsearch to all the systems that matter

##### Learn

• ###### Docs Learn how to use all of Elastic's products and features
• ###### Elasticsearch Labs Learn how to build with the latest features and abilities
• ###### Elastic Security Labs Understand the threat horizon and see the latest research
• ###### Elastic Observability Labs Explore what's next in monitoring and metric trends
• ###### Blog Read all of the latest company news from Elastic's blog

##### Connect

• ###### Community Join our community of developers on Slack, GitHub, and more
• ###### Events Attend your local meetups, workshops, and Elastic{ON}
• ###### Webinars Check out Elastic webinars and learn directly from our experts
• ###### Discuss Share tips, ask questions, and learn from other developers

##### Get help

• ###### Training Learn Elastic for free and expand your skills with our courses
• ###### Support Get expert advice on your Elasticsearch deployments for fast resolution
• ###### Consulting Drive success with custom support and consulting services

PricingDocs

Search

Start free trialContact sales

Blog

Company

* Solutions

* Stack + Cloud

* News

* Customers

* Generative AI

* Culture

Elasticsearch Labs

* Blogs

* Tutorials

* Examples

* Integrations

Security Labs

* Blogs

* Reports

* Tools

Observability Labs

* Blogs

Table of Contents

Table of contents

• Close

Herding Llama 3.1 with Elastic and LM Studio

By

Charles DavisonJulian Khalifa

September 12, 2024

• )Share on Twitter
• )Share on LinkedIn
• )Share on Facebook
• )Share by Email
• )Print

The latest LM Studio 0.3 update has made Elastic’s AI Assistant for Security run with an LM Studio-hosted model easier and faster. In this blog, Elastic and LM Studio teams will show you how to get started in minutes. You no longer need to set up a proxy if you work on the same network or locally on your machine.

About Elastic AI Assistant

Over a year ago, we put the Elastic AI Assistant in the hands of our users, empowering them to harness generative AI to resolve problems and incidents faster. _Learn more about the efficiency gains of Elastic AI Assistant__._

About LM Studio

LM Studio is an explorer/IDE for local large language models (LLMs), focused on making local AI valuable and accessible while providing developers a platform on which to build.

There are several reasons users consider local LLMs, but chief among them is the ability to leverage AI without giving up sovereignty over your or your company's data. Other reasons include:

• Enhancing data privacy and security

• Reducing latency in threat detection

• Gaining operational benefits

• Securing your organization in the modern threat landscape

LM Studio provides a unified interface for locally discovering, downloading, and running leading open source models like Llama 3.1, Phi-3, and Gemini. Locally hosted models with LM Studio allow the SecOps team to use Elastic AI Assistant to help with context-aware guidance on alert triage, incident response, and more. All without requiring organizations to connect to third-party model hosting services.

How to set up LM Studio

Download and install the latest version of LM Studio.

Once installed, download your first model by navigating to the Discover tab. Here, you can search for any LLM from Hugging Face and see a list of popular models curated by LM Studio. We will be setting up the Meta Llama 3.1 8B model for Elastic AI Assistant to leverage.

After you’ve picked your model, head to the Developer tab. Here you’ll start a Local Server for the Elastic AI Assistant to connect to. This allows applications or devices on your local network to make API calls to whatever model you run. Requests and responses follow OpenAI's API format.

Click Start Server, then select Llama 3.1 8B from the dropdown menu. You’ll want to ensure the Port is configured to 1234 (it should be like this automatically). Finally, turn on Serve on Local Network. This will allow any device on your network to connect to LM Studio and call whatever model you run.

How to set up Elastic AI Assistant

For this setup, we will run Elastic in Docker. We’ll reference the article “The Elastic Container Project for Security Research” from Elastic Security Labs. Instructions are also available on GitHub.

Elastic Docker installation

1. Install the prerequisites.

1. Change the default passwords.

1. Start the Elastic container.

Once Elastic is up and running, navigate to Stack Management > Connectors and create a connector. Update the URL value to include the host's IP address as 127.* address will not route. Fill in anything for the API key.

Now, in the security section, land on one of the subpages to open up the Elastic AI Assistant. Select the gear icon at the top right, and under the Conversation tab, turn streaming off.

Voilà! You can now use the model you previously loaded in LM Studio to power the Elastic AI Assistant and accomplish great things!

Elevate your security operations

This blog outlined how to easily elevate every security practitioner with Elastic AI Assistant configured with a locally hosted Llama 3.1 Model using LM Studio. This augments the security analysts' expertise by providing AI-powered guidance through triage, investigation, and response, improving productivity.

To learn more, check out our recent addition toElastic Security documentationon how to set up Elastic and LM Studio. For further information, refer to:

• The Elasticsearch Platform for SecOps

• Elastic Security | AI Assistant Demo

• Elastic Security SIEM

• LM Studio Documentation

• LM Studio 0.3.0 Update

_The release and timing of any features or functionality described in this post remain at Elastic's sole discretion. Any features or functionality not currently available may not be delivered on time or at all._

_In this blog post, we may have used or referred to third party generative AI tools, which are owned and operated by their respective owners. Elastic does not have any control over the third party tools and we have no responsibility or liability for their content, operation or use, nor for any loss or damage that may arise from your use of such tools. Please exercise caution when using AI tools with personal, sensitive or confidential information. Any data you submit may be used for AI training or other purposes. There is no guarantee that information you provide will be kept secure or confidential. You should familiarize yourself with the privacy practices and terms of use of any generative AI tools prior to use._

_Elastic, Elasticsearch, ESRE, Elasticsearch Relevance Engine and associated marks are trademarks, logos or registered trademarks of Elasticsearch N.V. in the United States and other countries. All other company and product names are trademarks, logos or registered trademarks of their respective owners._

Share

• )Share on Twitter
• )Share on LinkedIn
• )Share on Facebook
• )Share by Email
• )Print

Sign up for Elastic Cloud free trial

Spin up a fully loaded deployment on the cloud provider you choose. As the company behind Elasticsearch, we bring our features and support to your Elastic clusters in the cloud.

Start free trial

Follow us

• 
• 
• 
• 
• 

• About us About ElasticLeadershipBlogNewsroom
• Join us CareersCareer portalHow we hire

• Partners Find a partnerPartner loginRequest accessBecome a partner
• Trust & Security LegalTrust centerPrivacyTrade ComplianceEthics & Compliance

• Investor relations Investor resourcesGovernanceFinancialsStock
• Excellence Awards Previous winnersElastic{ON} TourBecome a sponsorAll events

About us

• About Elastic
• Leadership
• Blog
• Newsroom

Join us

• Careers
• Career portal
• How we hire

Partners

• Find a partner
• Partner login
• Request access
• Become a partner

Trust & Security

• Legal
• Trust center
• Privacy
• Trade Compliance
• Ethics & Compliance

Investor relations

• Investor resources
• Governance
• Financials
• Stock

Excellence Awards

• Previous winners
• Elastic{ON} Tour
• Become a sponsor
• All events

• Trademarks
• Terms of Use
• Privacy
• Sitemap

© 2026. elasticsearch B.V. All Rights Reserved

This website and all associated content, software, discussion forums, products, and services are intended for professional use only. No consumer use of this website or its content is intended or directed.

Elastic, Elasticsearch, and other related marks are trademarks, logos, or registered trademarks of elasticsearch B.V. in the United States and other countries.

Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries. All other brand names, product names, or trademarks belong to their respective owners.