Exploring Apache Logs with Essentia and R

Apache logs are some of the most troublesome logs being used today. They can come in all shapes and sizes and often come riddled with errors and missing data. While the benefits of analyzing these logs are enormous, most powerful analysis tools lack the necessary versatility. Essentia fixes that.

With the Essentia Log Converter you can transform any apache log into a format ready for analysis. You can use it to convert your logs and then stream them directly into the Essentia Preprocessor to ignore the irrelevant columns, clean the data, and perform simple analysis on it.

Read more

Data warehousing with AWS Redshift and Essentia

Amazon developed its Redshift service to accommodate data warehousing needs on a reliable, scalable platform.  With a highly efficient SQL engine that executes queries in parallel, users can gain insight into their data quickly.  But to access that power, the data first needs to be loaded into the service.

Going from raw data into a form that an application (in this case Redshift) is used, is one of the main strengths of Essentia.   Let’s focus one one the most common scenarios:  data is stored in its raw form on S3, which can be accessed by Redshift or any other relevant service.  Typically, the data is ‘dirty’, containing missing, irrelevant, or otherwise unneeded data.

Read more

Hadoop vs Essentia: Media Overlap Calculation

At AuriQ, we use Essentia to analyze digital marketing logs for a diverse set of clients. Some are interested in traditional attribution scores, others in lift metrics, but there is also an interest in brand exposure.

The issue is the following: We want users to see an ad on as many different web sites as possible. But which ‘groups’ of sites are most commonly visited? For simplicity lets focus on site pairs and count unique users who visit site A and B, site A, and C, etc etc, and then rank these in order of decreasing unique user counts.  We can then focus ad spend on those highest ranked pairs of sites.

Read more

Essentia Blog Active

Welcome to the Essentia Blog.  We’ll be adding lots of new content related to Essentia, Big Data, AWS, and more.  Our data scientists, developers, and customers all have great stories and insights to tell.  Stay tuned and check back over the coming days to this blog.