Browser docs

Visualize AWS VPC Flow Logs in Squey using Apache Parquet format

Jean-Baptiste Leonesio June 25, 2024 1 min read parquet logs network tutorial visualization parallel coordinates
Visualize AWS VPC Flow Logs in Squey using Apache Parquet format

Since version 5.0, Squey is able to import and export Apache Parquet files!

«
Apache Parquet is an open-source file format that stores data efficiently in columnar format, provides different encoding types, and supports predicate filtering. With good compression ratios and efficient encoding, VPC flow logs stored in Parquet reduce your Amazon S3 storage costs.
» ― AWS Blog

Let’s take advantage of the fact that AWS VPC Flow Logs can be natively stored in Apache Parquet format to seamlessly visualize our network and understand traffic patterns, identify security issues, audit usage, and diagnose network connectivity.

In AWS VPC console select a VPC, click Create flow log and set the following parameters:

  1. Send to an Amazon S3 bucket
  2. Custom format : standard attributes
  3. Log file format : Parquet

Then you can open in Squey as many Parquet files as you want in one go, as show in the quick video below:


May your findings be insightful! :)