parse_exception reason: Failed to derive xcontent

When inserting json to an elasticsearch index, this is a common error. It’s often the result of a simple error in the formatting. When inserting the following code for a demo, I came acrosss the error: POST /domaindb_index/domain/_bulk { “index”: { “_id”: 1 }} { “title”: “google.com”, “ip”: [“216.58.193.78”], “description” : “Words most popular search […]

high disk watermark exceeded on one or more nodes – elasticsearch

When starting elasticsearch many┬ápeople are getting the following error message “high disk watermark exceeded on one or more nodes”.   This is a message from elasticsearch, which by default comes up when the drive containing your elasticsearch index is more than 90% full. It’s not related to the size of the elasticsearch index, just the […]

21 handy elasticsearch queries

Elasticsearch is a powerful, fast and surprisingly user freindly data store and search tool. It’s not quite a database and it’s not quite a search engine, elasticsearch sits somewhere comfortably in between. For these examples, we’ll store some domain names, Ip addresses and related information as that is a lot more fun than traditional bookstore […]

elasticsearch curl query examples

A quick way to get results from elasticsearch, test your queries and make sure your data is being indexed correctly is to fire off some queries from the command line. Using curl, you can get this going quickly. View all your elasticsearch indices: curl -XGET http://localhost:9200/_cat/indices?v A quick search across all indices: curl -XGET http://localhost:9200/_search/?q=foo […]

elasticsearch export specific index mappings

It can be a little confusing learning how to export elasticsearch mappings for a specific index. Exporting mappings is a great idea, just like you’d export and keep your database schema when you’re using mySQL or some other relational database, it’s important to keep the building blocks of your data store. To do a full […]

Elasticsearch – getting started

For those coming from traditional SQL databases, or nosql databases, some of the terminology used by Elasticsearch may take some getting used to. Connecting to Elasticsearch can be done by connecting to the machine with Elasticsearch installed through localhost (127.0.0.1). The default port is: 9200, and access is granted full to everyone. Access is denied […]

Install and configure logstash forwarder (filebeat)

Most of the time you’ll want to use the ELK stack to process your logs from a separate server. Previously we used a tool called the logstash forwarder, however it’s now called filebeat instead. It makes sense to offload your log processing CPU work to another server so that it won’t interfere with the operation […]

ELK Stack Powered Analytics

Just what is the ELK Stack? The (ELK) stack consists of Elasticsearch, Logstash, and Kibana opensource applications. Each of these software packages is open source, yet managed by the open-source vendor Elastic. Even though they are separate applications, they’ve been designed and tested to work together magnificently. To help understand what the technology is all […]