Tail a log file through a browser

October 8, 2015 by richardvk

Filed under Web Development

Last modified January 30, 2018

When I started my career in the late nineties after graduating from university in computer science, I got a job as a junior support engineer at the largest corporate ISP in South Africa.

One of the functions of the job was network monitoring. However, there were no fancy tools or dashboards in my environment. Thats not to say they weren’t available at the time, we were just a bunch of (young and very inexperienced) Linux geeks and we preferred cool, raw tools like syslog. The most advanced it got was that we had syntax highlighting on a constantly running syslog file tail, so that as the logs ran up the screen it was ‘easy’ to spot issues that needed human intervention.

Of course we moved on from there eventually, but it was always something that stuck in my mind. Our team of 6 engineers was actually incredibly efficient at responding to alerts using such a basic method.

Syslog is still in common use in my environment today, but nowadays its all about importing into an ELK stack or similar, which clearly has huge benefits. But oftentimes, I really still wished I had a screen in front of my face with a tailing log file (it also looks cool and hardcore if thats not your line of business!)

So I set about scouring the net for examples of how I might achieve something like this today, but in a browser window that could be run on a big screen, instead of needing an SSH login with read permissions on the log file in question.

The solution I came up with was something called ‘Long Polling’. Basically, its a javascript client making a web call to a server with a predefined timeout. Either the timeout is reached and immediately a new, identical call is made, or the server responds (if it has picked up something new in the log file) and returns it to the browser.

You can find my code on Github.

In somewhat more detail, it works something like this:

  • When called for the first time, the browser script requests some content from the server with 0 (zero) as the current line count.
  • It then waits for the server, which should initially return the first 10 lines of the log file back to the browser, as well as the current max line number of the file.
  • The browser then absorbs the returned json data, puts it in a DIV and then immediately makes an identical call to the server again. However this time the returned last line number is included in the ajax call so the server knows what the last line was that the client received.
  • The server checks the current file length (number of lines) to determine if any new lines were written since the last call; if so it returns them immediately and the same process as above is followed.
  • If no new lines have yet been added since the last call, then the server script looks at the current file ctime to note when it was last updated.
  • It then goes into a loop, where every second it checks the file’s ctime and compares it to the ctime when the client made the call to the browser.
  • When the ctime is updated, the script retrieves whichever lines have been written and returns them to the client, who once again processes them, adds them to the current DIV content and then makes a new call to the server… and so the process continues!

I am sure nowadays you might use websockets to do something like this, in fact I havent revisited that idea (yet!), but this was still a pretty fun project and gave me pretty much exactly the results I was looking for.

Related Articles

                          Leave a Comment