News & Insights

Connamara Releases Logstash Filter for FIX as Open Source

Connamara, continuing a commitment to open source in capital markets, releases into open source a Logstash filter allowing DevOps teams to better analyze FIX Logs.

Connamara Systems is excited to announce the release of an open source FIX Protocol Logstash Filter!

FIX logs are often crucial in supporting trading systems.  However, in most scenarios, processing and parsing these logs are cumbersome for all but veteran FIX aficionados.  Logstash is a log parsing tool that excels at transforming raw and unstructured log messages into well-formed data that can be processed downstream by standard devops tools.  The FIX Protocol Logstash Filter makes it easy for developers to integrate Logstash with existing FIX applications.

Given a FIX log file that looks like this:

2015-08-26 23:08:38,096 FIX.4.2:DUMMY_INC->ANOTHER_INC: 8=FIX.4.2�9=184�35=F�34=2�49=ANOTHER_INC�50=DefaultSenderSubID�52=20150826-23:08:38.094�56=DUMMY_INC�1=DefaultAccount�11=clordid_of_cancel�41=151012569�54=1�55=ITER�60=20250407-13:14:15�167=FUT�200=201512�10=147�

2015-08-31 17:48:20,890 FIXT.1.1:DUMMY_INC->ANOTHER_INC: 8=FIXT.1.1�9=140�35=W�34=2�49=DUMMY_INC�52=20150831-17:48:20.890�56=ANOTHER_INC�22=99�48=.AQUA-W�262=golden_path_test�268=1�269=3�270=640754�272=20150831�273=17:48:20.882�10=070�
2015-08-31 20:48:26,536 FIXT.1.1:DUMMY_INC->ANOTHER_INC: 8=FIXT.1.1�9=189�35=W�34=5�49=DUMMY_INC�52=20150831-20:48:26.535�56=ANOTHER_INC�22=99�48=ITRZ21�262=req_A�268=2�269=0�270=0.01005�271=10�272=20150831�273=20:48:26.514�269=1�270=0.0101�271=2�272=20150831�273=20:48:26.514�10=123�

The FIX Protocol Logstash Filter plugin can read the FIX log as an input and turn it into something like this:

Logstash can take the output of the transformed FIX message and feed it into a time series data base like Elasticsearch.  Elasticsearch provides a very simple REST API that allows you to easily query FIX logs that have been processed by the FIX Logstash Filter.
Once in Elasticsearch, the FIX logs can be visualized with custom dashboards using Kibana, a visualization and analysis tool for time series data.

This combination of Elasticsearch, Logstash, and Kibana (ELK for short) can empower an operations team to dissect FIX logs on an enterprise scale and to identify anomalous activity at a glance.
Documentation and source can be found on Github.  Get started parsing your FIX logs today!  

Related Reads

A Sustainable Plan For Entrepreneurship: An Interview with Jim Downs on Connamara’s Longevity

Jim Downs founded Connamara in 1998 during a software development and tech boom along with soon-to-be giants like Google and.

QuickFIX/Go: The Latest Open-Source Library Release

QuickFIX/Go is a free, open-source library for the implementation of the Financial Information eXchange (FIX) protocol in the Go programming language..

Connamara Systems Announces SOC 2 Type II Certification

Connamara Systems is pleased to announce it has completed the Service Organization Controls (SOC) 2 Type II certification. For over.

New QuickFIX Engine Website Unveiled

New website for industry leading open source project Connamara Systems, co-originator and official maintainer of the industry standard open-source QuickFIX FIX library,.

Stephen A. Brodsky Joins Connamara Systems Board of Advisors

Connamara Systems is pleased to welcome Stephen Brodsky to its Board of Advisors.  Steve brings deep capital markets experience to.

FixSpec launch FIX Workspace: A new integrated tool for FIX API development

FixSpec is excited to announce the launch of a new connectivity App called FIX Workspace, which brings FIX logs, digital.

FIX Trading Community appoints Jim Downs to GTC Governance Board

Connamara Systems is pleased to announce Jim Downs, Founder and CEO, has been appointed to the Global Technical Committee Governance Board (GTC).

Connamara to Sponsor FIX Trading Community’s EMEA Trading Conference

Please join us at the FIX Trading Community EMEA Trading Conference next week.  Connamara is delighted to be sponsoring the all virtual.

Eris Innovations Migrates to AWS and Achieves Reliable Data Delivery and up to 50% Savings by Leveraging Connamara Systems and Real-Time Data from Bloomberg

Executive Summary By migrating its virtual machines (VMs) from a locally-hosted data center to Amazon Web Services (AWS), Eris Innovations.

Lessons in SHIFTing gears from bootstrapped startup founder Jim Downs

Jim Downs, CEO of Chicago-based custom trading software company Connamara Systems, has had a fascinating career path. Once an avionics.