[kwlug-disc] Scripting in Linux and unit.d

John Van Ostrand john at netdirect.ca
Tue Jan 10 09:15:11 EST 2012


Hi Colin,

Although it matters how you want to split the minecraft log, perl was meant for this very purpose. You might also look at logrotate if you want to split on date/size. It does it automatically, just create a file in /etc/logrotate.d.

If you want to split lines you may be able to do so depending on how minecraft writes logs.

It may be using syslog functionality so modifying /etc/rsyslog.conf may reduce the amount of stuff in the file or be able to split it based on log level (info, debug, crit, emerg.)

If it is using syslog and because of how it's using syslog I can't split it the way I want I write a filter and configure syslog to pipe to it through a named pipe. 

If you can't control how it writes logs then you'll have to manually process the file on regular intervals. Logrotate does this on size or date easily.

A perl script like this is a start

#!/usr/bin/perl
#
# Filter log file
#
# usage: myscript logfile

while (<>) {
  # match text in lines
  if (/failed login/) {
     # Do something with it
  elsif (/new user/) {
     # do something else
  }
}

Awk is also built with this type of functionality, but perl is as well suited and has much more flexibility.

To learn perl try http://learn.perl.org/. But you may want to skip the "installing" section since perl is installed on almost everything.



----- Original Message -----
> Well since i started this thread i didnt really expect it to take on a
> life of its own so i almost feel like this is off topic at this point.
> (dont poke the devs' :P) Anyways bash script looks like it would be
> lovely a current application that im looking to try and write is
> something to split a monolithic .log file for my minecraft server im
> guessing it would require grep and some other such stuff but all of
> this massive central log is just ugly and painful to sort through.
> Server isnt even hugely popular yet and its already an 11mb log file.
> Its fairly important that I can search through it for keywords and
> retain the logs for some time to help sort out disputes and track
> staff abuse.
> 
> 
> This isnt the only application that i've needed through my time on
> computers but this is the one i need right now.
> 
> 
> On Mon, Jan 9, 2012 at 11:56 AM, Paul Nijjar < paul_nijjar at yahoo.ca >
> wrote:
> 
> 
> 
> On Sun, Jan 08, 2012 at 12:19:25PM -0500, Darcy Casselman wrote:
> > A gentleman named Albert O'Connor is interested in starting up
> > WatPy,
> > a local Python user group. He's tentatively scheduled Wednesday,
> > February 8 as the date of an initial organizational meeting. I don't
> > have any other details, but I'll let him know there's interest here.
> 
> It may be helpful to distinguish "User Group" from "Beginner
> tutorial/course". That has been my experience with KWLUG, anyways.
> 
> As some of you know I used the KW Freeskool infrastructure for the
> Linux Literacy course. That may not reach the demographics you want
> (and you may not want to be associated with their political
> affiliation) but they can offer resources if you are interested.
> http://kwfreeskool.wordpress.com
> 
> I think both Dive into Python and Learn Python the Hard Way lack
> sufficient exercises to be really good training materials, so you will
> have to come up with your own. An interesting mix might be to use one
> of these books in conjunction with the Project Euler
> ( http://projecteuler.net ) series of problems.
> 
> Another book that looks promising is "Building Skills in Python":
> http://www.itmaybeahack.com/homepage/books/python.html
> 
> You could also turn to open courseware. Here is a StackExchange
> post with some good resources:
> 
> http://stackoverflow.com/questions/4463098/where-can-i-find-a-good-online-python-course
> 
> All of this depends on your audience. It makes a big difference
> whether you are trying to get experienced programmers up to speed in a
> language, or whether your are trying to get people to learn
> programming with Python as the vehicle.
> 
> Sorry for the wall of text.
> 
> - Paul
> 
> 
> 
> 
> >
> > Darcy.
> >
> > On Sun, Jan 8, 2012 at 12:09 PM, Michael Savage < msavage at golden.net
> > > wrote:
> > > I am interested.
> > > Mike
> > >
> > >
> > > On 08/01/2012 3:33 AM, Bob Jonkman wrote:
> > >>
> > >> Charles McColm and I were recently bitten by the
> > >> "We-want-to-be-Python-programmers" bug. On Friday at Ubuntu Hour
> > >> we met a
> > >> fellow from London (ON). Sadly, my memory has lost its parity
> > >> bits and I
> > >> don't recall his name. Maybe "Mars" on the Ubuntu IRC channel?
> > >> Anyway, he's
> > >> a professional Python programmer, and he said he might be willing
> > >> to come
> > >> back to guide some of us newbies towards a life of Pythony
> > >> goodness. He
> > >> suggested we start with Mark Pilgrim's "Dive Into Python" or Zed
> > >> A. Shaw's
> > >> "Learn Python The Hard Way".
> > >>
> > >> Is there interest in learning Python within the ranks of KWLUG?
> > >> All we
> > >> need is a project to work on... The KWPUG group seems to be
> > >> defunct, but we
> > >> can strike out on our own.
> > >>
> > >> --Bob.
> > >>
> > >> Dive Into Python http://www.diveintopython.net/
> > >> Learn Python The Hard Way http://learnpythonthehardway.org/
> > >> KWPUG http://kwpug.org
> > >> Ubuntu Canada IRC channel irc:// irc.freenode.net/#ubuntu-ca or
> > >> http://webchat.freenode.net/?channels=ubuntu-ca
> > >>
> > >>
> > >>
> > >> John Van Ostrand < john at netdirect.ca > wrote:
> > >>
> > >> Hi Colon,
> > >>
> > >> Scripting in Linux relies heavily on the vast array of programs
> > >> either directly or indirectly.
> > >>
> > >> Bash is the command shell and is commonly used for scripting,
> > >> especially in system start up. It uses external programs heavily.
> > >> I
> > >> usually start with bash and if it proves poor for the task I move
> > >> to
> > >> perl by massaging the script.
> > >>
> > >> Perl is a very flexible and powerful scripting language and it
> > >> borrows a lot of syntax from bash, awk and grep. So learning bash
> > >> helps with perl. It works well for parsing text files.
> > >>
> > >> Python seems to be one of the more recently popular scripting.
> > >> With
> > >> add-ons it's used for Fedora admin Gui programs. It also handles
> > >> threading well and has lots of other add-on modules.
> > >>
> > >> There are lots of other options but these are the common
> > >> workhorse
> > >> apps for system-level scripting.
> > >> Bash and the common linux commands are worth getting familiar
> > >> with.
> > >> Regular expressions are also commonly handy. These are the basics
> > >> that I findf handy all the time.
> > >>
> > >> What are you trying to do?
> > >>
> > >> ------------------------------------------------------------------------
> > >> *From*: kwlug-disc-bounces at kwlug.org
> > >> *To*: KWLug Discussion
> > >> *Sent*: Sat Jan 07 18:48:02 2012
> > >> *Subject*: [kwlug-disc] Scripting in Linux and unit.d
> > >>
> > >> I'm not a programmer but I do understand some fundamentals and
> > >> looking to learn some scripting so that I can automate some
> > >> tasks.
> > >> What would be a good way to learn this stuff.
> > >>
> > >>
> > >> _______________________________________________
> > >> kwlug-disc mailing list
> > >> kwlug-disc at kwlug.org
> > >> http://kwlug.org/mailman/listinfo/kwlug-disc_kwlug.org
> > >>
> > >
> > >
> > > _______________________________________________
> > > kwlug-disc mailing list
> > > kwlug-disc at kwlug.org
> > > http://kwlug.org/mailman/listinfo/kwlug-disc_kwlug.org
> >
> > _______________________________________________
> > kwlug-disc mailing list
> > kwlug-disc at kwlug.org
> > http://kwlug.org/mailman/listinfo/kwlug-disc_kwlug.org
> 
> --
> http://pnijjar.freeshell.org
> 
> 
> - Paul
> 
> 
> 
> _______________________________________________
> kwlug-disc mailing list
> kwlug-disc at kwlug.org
> http://kwlug.org/mailman/listinfo/kwlug-disc_kwlug.org
> 
> 
> _______________________________________________
> kwlug-disc mailing list
> kwlug-disc at kwlug.org
> http://kwlug.org/mailman/listinfo/kwlug-disc_kwlug.org

-- 
John Van Ostrand 
CTO, co-CEO 
Net Direct Inc. 
564 Weber St. N. Unit 12, Waterloo, ON N2L 5C6 
Ph: 866-883-1172 x5102 
Fx: 519-883-8533 

Linux Solutions / IBM Hardware 




More information about the kwlug-disc mailing list