Chris Musty's picture

Hi all,

In another thread it was mentioned that a syslog server would be a cool idea. I use them alot to monitor anything from routers to servers and they can be a powerful tool to keep you up to date when you manage many sites or even if you have many devices to monitor.

I am contemplating creating one from TKL LAMP stack (if I use a database) and would like to hear from anyone interested in this feature. Please add your RFF here and I will start researching!


  • Highly configurable email alerts
  1. Delivery rules based on IP address
  2. alert level
  3. increase in volume
  4. abnormal conditions
  • and SMS alerts (would require a provider or GSM modem)
  • Customisable error levels
  • auto archiving features
  • auto report generation
  • up and downlink speeds
  • Tons of reports
  • Design web GUI from scratch
  • Heaps more but I want to hear from others
Liraz Siri's picture

Hi Chris, great of you to take the initiative on this. I think a TurnKey log collection appliance is a great idea, though I haven't really thought too much about this problem yet so I don't have too many specifics to add. Come to think of it, even if I had thought about that might not even matter because the ideal configuration may depend on the circumstances of your usage scenario anyhow (e.g., how many systems you are collecting logs from and what you are collecting logs for, etc.). Or not. Good designs can be generic.

Regarding the web GUI, looks pretty sweet. Even if the implementation doesn't fit for the usage scenario you are thinking about, there might be a few interesting ideas in there...

Chris Musty's picture

I must admit I had only considered my usage scenario when deciding to do this, mainly bacuase I had already planned this a while ago. I am not sure how you would even define my usage scenario as I have particular needs. Saying that when I get into it I will design for what I require and if anyone else finds it useful then great. Failing that there could be several versions.

I think the greatest point with this is that there is no easy "turnkey" solution for this. The first one  I created (still running!) took over a day to get together and now it would probably take between 4-8 hours to complete. This is a screaming need for a standard appliance that can be just launched from an ISO then mabey a quick config screen and your running.

Eventually it may evolve into a multiple type appliance that you can choose during setup. Lets see what happens.

Thanks for the link Liraz, interesting...

Chris Musty


Specialised Technologies

Selim Nart's picture

I was wondering if you ever setup a syslog turnkey appliance?





Jason Lehman's picture

We have a ton of Cisco hardware that needs monitored.

Just wondering, any word?




Jeremy Davis's picture

But I have just created a blueprint for it so it doesn't get forgotten.

Chris Musty's picture

I have researched a little but have not done any real dev work. As with so many other people here - I am very busy. I have 2 major projects on at the moment so when they are delivered I may delve into this. Sorry if I got anyones hopes up! 

Chris Musty


Specialised Technologies

Chris Musty's picture

Found this on my web travels - does anyone have any experience with it?

Another project I am toying with is monitoring for windows desktops eg Smart Data, Event logs, Software Installs etc any ideas or thoughts?

Chris Musty


Specialised Technologies

Jeremy Davis's picture

But just as a replacement for the default logger in it's default Ubuntu configuration (apparently it has much better performance under OpenVZ). I haven't tried to take to take advantage of any of the other advanced features (nor did I even realise that it had them)!

Jeremy Davis's picture

In my random online travels I just came across this interesting piece of work called PartyLog2. Looks like it's like a log collection/monitoring server just like you were/are looking for (built on top of TKL Core v11 already!)

The software it uses is called Graylog2 which has a catchy byline of "Manage your logs in the dark and have lasers going and make it look like you're from space."!!! I like it already! :) It also has a (separate) WebUI and even a custom log format thingy called GELF (which I won't even pretend to fully comprehend, but I'm sure it's good!)

I don't know anything more than that about it (which obviously isn't much!) but IMO seems worth a look. I've just posted on the devs SF page so hopefully we'll hear from him over here soon! I'm really hoping that he'll want to work with us on this one.

Chris Musty's picture

I knew someone had already done it, I just knew it!

Now to find some time to play around with it...

Chris Musty


Specialised Technologies

Jason Lehman's picture

My initial thoughts...

Install is a little different, as Partylog2 is only available as an ISO download.

I'm setting this up in VMware's vCenter, so since its an ISO; I have to give vCenter all the details about this operating system. (which I dont know all the details or requirements)

Had to take some guesses whether to tell VMware if this was to be a 32 or 64 bit server.

I guessed 64 bit.

The build of Graylog2 is slightly outdated (its running 0.9.5P2), (the new build has some major benefits) & there is no simple way to update it. I tried following an upgrade guide to get to the latest version of Graylog2 here...

But I ended up trashing my Partylog2 / Graylog2 server & had to start over. Not sure what i did wrong. It may just be a little too complex for me. Better update/upgrade method needed.

So, now Im back to the original Partylog2 / Graylog2 build that Jeremy linked above.

I will setup some devices / servers to log to it & will see how it works.

I will update my findings.



francajluis's picture

Hello everyone.

Glad you like it.

Im updating Partylog to Graylog2_0.9.6 and tk11.3

I will announce once its ready :)

francajluis's picture

Greetings everyone,

Partylog2 has been updated with the latest and more enterprise edition
of Graylog2.

Features of Partylog2:

   Graylog2 Server (v0.9.6)
   Graylog2 Web Interface (v0.9.6)
   mongodb v2.0.2
   elasticsearch v0.18.7
   ruby v1.9.3

You may download and test this release from:

And you can find more information in:




Jason Lehman's picture

After starting over again (this time for the latest release 0.9.6) I had issues getting the graylog2-server service to run. I restarted the machine twice, I would get the message in the the Graylog2 web interface, "It seems like your Graylog2 server is not running." Odd, that I could hit the webinterface.

I manually stopped & started the graylog2-server service & it now seems fine.

I hope that a TKLPatch is created & an official TKL appliance is created for this. There is potential here. I can't really say too much else at this point; as i have to let it collect data & see what I can get out of the appliance.

Keep up the good work Jose. I will update my process.


francajluis's picture

Thanks for the info.

Updated the release. I will apreciate if you look into it one more time if you have the time.

Jason Lehman's picture

I still got the same warning message imediately after configuration. Maybe I should have given it more time, but I restarted the server. After the restart everything was fine. No more need to manually stop/start any services.

I'm still looking through documentation & searching the web for the best way to get this configured.

I am getting devices to log to it with no problem. Thats the easy part. Now the hard part, get something out of all these logs. Hopefully the community will share what they are doing, so we dont all have to reinvent the wheel.

We are interested in monitorying Active Directory servers, SQL, Web (IIS & Apache), Cisco Network devices and a few others.

Thanks for fixing


Jason Lehman's picture

Sorry for the late update.

But after allowing a few servers (5 active directory servers) to log to the Partylog server for 2 weeks, I decided it was time to try to figure out what I could get out of these logs. The 50GB root partition I gave this server was full & Partylog was no longer functioning. At the same time, our network admin (the one who requested a syslog server) came in & said he found another solution that is working well for him. I ended up deleted the Partylog server. I will keep it on my radar for future requests & revisit some time in the future to see how it has improved. Keep on working on this, there is still a lot of potential.


Nick's picture

Indeed very odd.. The webservice is running, but no messages coming in. I don't have this problem on another server, which contains exactly the same version installation.

After a complete reinstall of the iso, i get the same error.


A service stop / start doesn't help here.

Jeremy Davis's picture

But is it possible for you to provide a TKLPatch though (as I detailed on your SourceForge forum). Even if you haven't got time or energy to do that, if you could share your install/config notes that would be enough for someone else to build the patch.

Then hopefully we could get this into the next official TKL appliance release.

francajluis's picture

Jeremy, I will look into it.

Thanks for your support.

Jeremy Davis's picture

Perhaps it's documented somewhere but I hadn't seen it previously... Have a look here for link and usernames/passwords for a public/demo Graylog2 instance. Not sure how official it is, but regardless I think it could be handy for those that want a sneak peak.

Kristoffer Bouchard's picture

I've been testing the Partylog2 for a few days on my test server (OS X Host, Virtualbox) and I can confirm some of the issues identified here.  Occasionally on boot the graylog service does not start even though you can log in, you need to restart the service or the VM (sometimes multiple times) before it works.  I've also experienced the mongodb lock file issue, where you get 502 error.  I attempted a mongod --repair of the database without success, I reinstalled the VM.

Overall I am very excited about this project, I would like to have a small, easy to deploy syslog server in a nice VM package such as turnkey linux, I really hope this test project goes live and becomes part of your library of great VMs.  Keep up the good work.

Side note- I am not overly experienced with syslogd within your other Turnkey VMs.  Can you steer me towards a cheat sheet to set your up Turnkey Core or Turnkey Torrent server to report their syslog to Partylog2? Is there a Webmin option or do I need to commandline edit some config files? Thanks for your help.

Kristoffer Bouchard's picture

Just a follow-up question, I have successfully configured 2x FreeNas devices, and 1x PFSense box to report to Graylog2/Partylog2. However I have 2x Turnkey VMs running 1 Core, 1 Torrent server and I cannot figure out how to configure the rsyslog.conf file to report to the syslog server. Some details below:

1. IPs of all servers are known

2. Rsyslog.conf- I have attempted to add a line

*.* @@192.168.1.xx:514 (send the syslogs to that IP, on that port using UDP)

3. Restarted rsyslog server, and rebooted VM

4. However the syslogs are not reporting

Any assistance is appreciated, or plse steer me towards another forum post.

Luciano's picture

Hi Kristoffer, be so kind to explain briefly how you have done to lead the syslog from pfSense to GaryLog? I try to implement it with Mikrotik, I point this to the syslog GaryLog did not get any information that can be displayed on the webgui of PartyLog2.

Of course I appreciate your response.

Kristoffer Bouchard's picture

I set up pfsense by adding my graylog server as a remote syslog server using the webgui under system logs settings

Sean McGerty's picture


Big fan of the project thanks everyone. Yes I'm deploying / testing PartyLog2_0.9.6_r1.iso, and I'm seeing in almost all cases that the graylog2 web service starts but the graylog2 service doesn't. Have been getting in and doing it manually at the moment, but I'd like to roll one of these out soon :)


Jeff McNamara's picture

My first two builds of Graylog were working ok for testing, collecting SIP and ISDN logs from media gateways until a large volume of traffic was moved to those devices, one test box only had 1Gb of ram, the other had 2Gb and the mongoid.yml index settings batch size was at default 4000/1 sec, we were only trying to match the output against what we have, if it didn't match there was no need to load test, both test boxes crashed with Graylog ooops something went wrong error.
Failed to save the recipe and was unable to get Graylog working exactly as it was the first time, the media gateways stuff 1220 bytes/packet into UDP 514 by default and can send 5,000 messages/second.

Decided to load Partylog2, same two test boxes were up in 10 minutes collecting logs, test boxes are only Dell 320's w/E4400, added 4Gb ram (only room for 2 sticks, won't mix ram sizes, whether it uses the top 1Gb or not).
In the past 48 hours the primary box has collected 1.5 million logs, after we dropped the level to 1, flow only messages, still not seeing certain messages and looking into the possibility of Graylog dropping messages with such a large packet size. RFC 3164 states packet size must be 1024 bytes or less, unless it's been revised or the equipment manufacturer is making a false claim that its RFC 3164 compliant I don't know yet.

At any rate, Partylog2 worked flawlessly right out of the box under intense conditions in underpowered test devices that should be 8 cpu cores and 8Gb of ram, the only alternative that works for so much data is Splunk, which is awesome but comes with a pricetag I'm not sure will get approved for funding.

We are going ahead with testing for 7 days of heavy traffic, then setting the logging level of the media gateways back to 5, a lot more data. When its all done, we'll put a final build for Graylog on ESXi hosts for which getting server space is going to be a premium since this solution can't reside with any other servers under the same load.
My hats off and many many thanks a lot to TurnKey and Partylog2!


Thanhd's picture

Those who has it working. Can you tell me if there is some trick to this! I have tried many times with no luck.

After I install using the latest ISO and can log into the webgui and see the messages for localhost. I set of of our other systems to send the log over to partylog2. However I'm not seeing anything show up in the webgui!

I have tried installing IPTraf and it is showing logs hitting the partylog2 vm yet they do not show up???

What I'm I missing???

francajluis's picture

CHANGELOG: Changes from r1 to r2:

  • Service startup fixed.
  • Graylog2 Server and Graylog2 Web Interface updated to version 0.9.6p1-RC2
  • Change upstart scripts to sysvinit (due to Turnkey Core 12 being based on Debian instead of Ubuntu)

I'd be happy if you guys can test this version and report any problems.

Thanks for your feedback, its always appreciated.

francajluis's picture

Thanhd's picture

Thank you for the new version, it's looking good so far. I can see the logs coming in unlike the previous version.

Snickasaurus's picture

Hello my fellow nerds/geeks. I came across this thread this morning and as I read through I hoped to see a more recently dated post at the bottom but did not. Is this something that was forgotten or In my joyous times of insanity have I been stuck trying to many other flavors of TKL that I skipped over the logging one?

Hope I don't offend anyone by resurrecting this old thread!


What I'm trying to log:

(7) tkl vm's running on XenServer

(2) boxes running FreeNAS

(2) hardware dedicated debian  servers

(2) Asus routers running DD-WRT

(1) Linksys 48 port gigabit switch

(1) 30 year old girlfriend that thinks she's a queen*

(1) 148lb gratedane that things he's a puppy*



Jeremy Davis's picture

FWIW I have lodged a Candidate Request for Graylog2 on the TKL Tracker Dev Wiki. Although TBH I'm not sure whether it meets your specs (I don't know enough about Graylog2).

I'm fairly sure that your last 2 will require significant tweaking! :)

Snickasaurus's picture

Hopefully it will get some traction and turn into another addition to the ever growing TKL library. I think this weekend I'll download the Core and setup my own logging system just to see what I can accomplish. Currently I have a Debian vm running and will try logstash first then move around a list I've compiled from Google searches. Perhaps I can contribute something here instead of lurking in the shadows on the forum.   ;-)

Jeremy Davis's picture

It'd be great if you wanted to lead the effort on this! If you come up with something that fulfills your needs then it could be the basis for an appliance! That Graylog2 software looks like it could be a goer? But IMO it doesn't have to be that...

If you can get your head around TKLDev then perhaps you could even build the appliance - or at least the bones of it?!

Snickasaurus's picture

I'm interested, years later, to see if anyone of you that posted above ever found a solution to this and can we agree a TKL solution would still be great for logging?

Jeremy Davis's picture

AFAIK this never went anywhere unfortunately. However, I would love to see an appliance in this space.

I certainly agree that a centralised logging server would be great!

If you have any further up to date feedback and input, I'd love to hear about it. I'm not sure if/when we'll get a chance to push this ahead internally. But if you have any success, then I'd love to hear about it.

As per always, I'm more than happy to coach you (or anyone) if you want to have a go at creating a new appliance. Even if you don't create an appliance, even documenting any success you have/had would be awesome!

Snickasaurus's picture

I consider myself an above average *nix user but I learn by doing and not so much reading. Some 'man pages have actually put me to sleep while trying to interpret them. However, I do have two servers that aren't doing anything right now and would love to get something going in the way of a log server and web front end that is stable and easily updateable. I'll spin up several TurnKey Core's and go as far as I can (documenting along the way). Should I post back here with issues or start a new thread and link back to this one or just keep everything here?

Thanks for the reply.

Jeremy Davis's picture

If you get something up and running on TurnKey v15 that you're happy with, and only includes open source software, that would certainly lower the bar to getting it added as a new appliance.

Seeing as this thread is so old, I suggest that you start a new thread for your "log server" adventures. Although a post here with a link to the the new thread might be nice for any that would like to follow the progress?!

If you keep in mind, that once you have things running on Core as you'd like, then the final step is creating build code, then that might help the transition from PoC on Core, to appliance build code. Essentially, every step needs to achievable via a non-interactive script (or an overlay file).

When it comes time to create the new appliance, you'll need a TKLDev server (our dev environment). A local VM is fine (FWIW that's what I use). It's fairly well documented (there's also a "meta-doc page" with links to all the various resources). Although you probably won't want to read all that! :)

Also, some of the docs relate to the previous v14.x version and haven't been updated. For instance, the sandbox doesn't quite work as it used to. Also the deck layers also don't work the same as they used to unfortunately. I need to at least update the docs to note those changes, or ideally fix our v15.x TKLDev tools so they work like they used to... (although not sure when I'll get a chance to do that).

Also, something worth noting is that there are lots of "best practice" things and conventions that we use that aren't very well documented at all. So please feel free to keep me in the loop on what you're up to and I'm more than happy to provide advice along the way.

Also, as I know the appliance library quite well, so if you hit an issue, then chances are there is already a known way to work around it (or I'll at least have some ideas). Also there may be code snippets which you could rob from an existing appliance. So please keep in touch and I'll help out where possible.

Rob tisdell's picture

I'm in agreement that a Turnkey Syslog Server Appliance is needed. I am currently in the process of finding a good fit. I'm willing to help out in any way that I can with the development of creating a Turnkey solution. Let me know what I can do.

Jeremy Davis's picture

Be great to have others involved too Rob.

From my perspective, I don't have time ATM to be too heavily involved myself, but am more than happy to try to assist where I can. I suggest that everyone try to work publicly as possible so that we can all bounce ideas around and gain from others' experiences.

I guess for now, just sharing thoughts and ideas is as good a place to start?!

Snickasaurus's picture

Tomorrow I'm going to start going through all the documentation of setting up the dev environment. I just hooked up a Dell server I had laying around to test with. Rob, I'll get with you over the next day or two so we can get started.

And as we discussed earlier Jeremy I'll start a new thread.

Jeremy Davis's picture

I look forward to hearing how you go with it all. :)

Alex DiMarco's picture

Any progress on this?  
thompsonmax465_1138580's picture

Hello everyone. I am new here. Interesting thread, thanks for the information. 

Chris Musty's picture

Hey all,

It seems a lot happens in 9 years!

SInce then I have deployed Graylog a billion times and it is awesome. Absolutely no point reinventing the wheel here.

Chris Musty


Specialised Technologies

Jeremy Davis's picture

Hey Chris, I think both of those recent posts were spammers. Thanks for dropping in anyway. Seeing as this thread is so old, I'm going to lock it for now. Take care mate.