LINQ To Logs And Traces
Some interesting features –
- Allows use of LINQ on raw event sources
- Enables use of Reactive Extensions on real event sources with support for multiplexed event sequences (single sequence containing events of different types in order of occurence).
- Possible to provide single query across multiple sources, with same API for both real-time and past history
- On historical log/trace files, multiple queries can be performed in one read – for e.g. count all “Warning” events, match “Begin and “End” events and calculate average duration of each activity
You can either use LINQPad for one-off analysis or build .NET applications for building monitoring applications. In LINQPad, the experience of Tx is as if all the events were in a database.
The release provides 4 different NuGet packages:
- Tx.Core – common components not specific to a particular tracing format
- Tx.Windows – Support for Event Tracing For Windows, Event logs, Performance counters from files and real-time counter API, IIS Text logs in W3C format
- Tx.SqlServer – SQL Server extended events
- Tx.All – A convenience package with all of the above
Note that Microsoft also advises when not to use Tx as well -
- When there are no real-time feeds involved and data is already in memory or in a single file that is easy to parse, the guidance is to use LINQ-To-Objects instead of Tx.
- When there are real-time feeds but each feed/file contains only a single type of events, use only Reactive Extensions
The tool has been in use internally at Microsoft in the WCF and Service Bus teams and is now open for all .NET developers to use for their own projects. You can have a look at the documentation to get started.
Real time data ingestion and event analysis
Thanks for writing this post. There are a lot of emerging requirements for real-time monitoring and logging analytics of events. There is a whole set of solution options in the open source arena. The ability to collect and integrate events in real-time from Windows/Microsoft systems is a potential weak link in existing tools.
You might find the link below interesting.