Run real-time search and vector analysis on large-scale data
You can find specific words in billions of logs or documents and perform semantic vector search within a single system. Elasticsearch is an engine for storing and analyzing data in distributed environments, optimized to handle production workloads. This allows you to perform various tasks, such as search, monitoring, and generative AI integration, on one platform.
Combining search and analysis
Elasticsearch serves as an extensible data store and vector database, going beyond a simple search tool. It performs near-real-time search on large datasets and supports vector search, enabling integration with generative AI applications. It is designed to handle various use cases, including logs, metrics, application performance monitoring (APM), and security logs. As the foundation of the Elastic open stack platform, this engine meets production requirements for speed and relevance.
Setting up a local development environment
The simplest way to start is by creating a managed deployment through the Elasticsearch Service on Elastic Cloud. If you prefer to install and manage it yourself, you can download the latest version from elastic.co/downloads/elasticsearch. For local development or testing with Docker, running the start-local script sets up Elasticsearch and Kibana together. This configuration provides a one-month trial license, after which it converts to the Free and open - Basic license. Note that this local setup is not suitable for production deployments because HTTPS is disabled and only Basic authentication is used.
Data ingestion and search methods
Data is sent as JSON objects (documents) via the REST API for indexing. Unstructured text, numerical data, and geospatial data are all stored efficiently and support fast search. Log or metric data with timestamps is added to data streams composed of multiple auto-generated backing indices. If you add a single document, the index is created automatically if it does not exist, and it becomes immediately searchable across all nodes in the cluster. To add multiple documents at once, use the _bulk API, which requires Newline Delimited JSON (NDJSON) format.
Pre-use considerations
Elasticsearch is a Java-based distributed system that uses Gradle as its build system. To build from source locally, you can run the ./gradlew localDistro command. This tool is suitable for large-scale data environments requiring search, vector analysis, and AI integration. However, the start-local script for local development has limited security settings and should not be used in production. You must verify license policies and compatibility for specific versions in the official Elastic documentation and subscription information.
