Cookies help us display personalized product recommendations and ensure you have great shopping experience.

By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData CollectiveSmartData Collective
  • Analytics
    AnalyticsShow More
    data analytics and truck accident claims
    How Data Analytics Reduces Truck Accidents and Speeds Up Claims
    7 Min Read
    predictive analytics for interior designers
    Interior Designers Boost Profits with Predictive Analytics
    8 Min Read
    image fx (67)
    Improving LinkedIn Ad Strategies with Data Analytics
    9 Min Read
    big data and remote work
    Data Helps Speech-Language Pathologists Deliver Better Results
    6 Min Read
    data driven insights
    How Data-Driven Insights Are Addressing Gaps in Patient Communication and Equity
    8 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-25 SmartData Collective. All Rights Reserved.
Reading: Monitoring a System
Share
Notification
Font ResizerAa
SmartData CollectiveSmartData Collective
Font ResizerAa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Uncategorized > Monitoring a System
Uncategorized

Monitoring a System

Editor SDC
Editor SDC
5 Min Read
SHARE
I’ve been using Python with IbPy, rather than Matlab. The code I post will most likely be in Python going forward. I like it because it’s more natural, ex. default and variable arguments are concise, no semicolons, dictionaries; it’s open source so there are more libraries and it’s free (Matlab requires a “license server” for instances running in parallel, for example); and it’s easier to organize code with better package management, simple classes, and files are more flexible.

Anyway, I like to know what’s going on with my system when I’m away. The system writes out a log of errors, trades, finished processes, etc, and also emails me important events like Trader Workstation shutting down, and executed trades. These go to my Blackberry wherever I am. To send emails from/to a Gmail account…

I’ve been using Python with IbPy, rather than Matlab. The code I post will most likely be in Python going forward. I like it because it’s more natural, ex. default and variable arguments are concise, no semicolons, dictionaries; it’s open source so there are more libraries and it’s free (Matlab requires a “license server” for instances running in parallel, for example); and it’s easier to organize code with better package management, simple classes, and files are more flexible.

Anyway, I like to know what’s going on with my system when I’m away. The system writes out a log of errors, trades, finished processes, etc, and also emails me important events like Trader Workstation shutting down, and executed trades. These go to my Blackberry wherever I am. To send emails from/to a Gmail account, which I like to use because it has a lot of storage and can be searched easily, I had to write a little script (code at the bottom) to add to the Python logging library since Gmail uses a special kind of authentication not supported by the standard library.

Here’s how I use it:
import logging
logging.config.fileConfig(“/yourdirectorystructure/logging.conf“)
logging.debug(‘Live trading online’)
logging.critical(strategy_name+’ ‘+signal+’ ‘+position_size+’ ‘+etf)
What that says is: load Python’s logging library (which does most of the work for me); then configure it according to my preferences in the file logging.conf; then as an example send a debug-level message ‘Live trading loop starting’; and then a critical-level message with the strategy name, buy/sell signal, position size, and ETF name. These two messages are taken out of context from one of my strategies, so the variables don’t reference anything here. The first is called when I turn on the system, the second is called whenever a trade is executed.
It looks like this in the inbox:
logging.debug(msg) sends a debug(low)-level message to be logged, in my case it prints to a console window and writes to a file. logging.critical(msg) sends a critical(high)-level message, which goes to the console and file, and also gets sent to my email address.
It used to be frustrating to wake up in the morning and see that my system had crashed at some unknown point in the night for some unknown reason, but now I know what’s going on, when it’s happening, and why.
Python’s logging library isn’t documented very well, especially the email handler and file configuration, so I thought sharing these two parts would be useful. Here’s my configuration file, from above (modify it to set the directory where you want to store logs), and the custom Gmail email handler (modify it for your own username/password). The overall logging system works well and adds the minimum amount of superfluous code.
I’m interested in knowing how others have approached the problem of monitoring and logging.

More Read

The Inevitable Wolfram|Alpha Problem: Semantics
Facebook ‘Hilarious Video’ Phishing Attack is Spreading Quickly
Overcoming Objections to a Data Governance Program
Empowering Business Users To Embrace Change
The Outline for Your Next IT Strategy Meeting
TAGGED:pythonsystem architecture
Share This Article
Facebook Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

data analytics and truck accident claims
How Data Analytics Reduces Truck Accidents and Speeds Up Claims
Analytics Big Data Exclusive
predictive analytics for interior designers
Interior Designers Boost Profits with Predictive Analytics
Analytics Exclusive Predictive Analytics
big data and cybercrime
Stopping Lateral Movement in a Data-Heavy, Edge-First World
Big Data Exclusive
AI and data mining
What the Rise of AI Web Scrapers Means for Data Teams
Artificial Intelligence Big Data Exclusive

Stay Connected

1.2kFollowersLike
33.7kFollowersFollow
222FollowersPin

You Might also Like

Apache Spark
Big DataData ManagementExclusiveNewsSoftware

Apache Spark Pitfalls: The Limitations of the Big Data Processing Giant

5 Min Read
analytical hub architecture
AnalyticsBest PracticesBig DataData QualityITModelingPredictive Analytics

5 Principles of Analytical Hub Architecture (Part 1)

3 Min Read
python best language for big data
Artificial IntelligenceBig DataBlockchainBusiness IntelligenceData ManagementData MiningData QualityData ScienceData VisualizationHadoopITMachine LearningUnstructured Data

Here’s Why Python Is The Top Programming Language For Big Data

6 Min Read

5 Principles of Analytical Hub Architecture (Part 2)

2 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

ai is improving the safety of cars
From Bolts to Bots: How AI Is Fortifying the Automotive Industry
Artificial Intelligence
ai chatbot
The Art of Conversation: Enhancing Chatbots with Advanced AI Prompts
Chatbots

Quick Link

  • About
  • Contact
  • Privacy
Follow US
© 2008-25 SmartData Collective. All Rights Reserved.
Go to mobile version
Welcome Back!

Sign in to your account

Username or Email Address
Password

Lost your password?