[kasperl]

Ok, I need a quick way of doing this bit of pseudo code in C, C++, PHP, or whatever you can think of that works under XP and has a free compiler.

while(file(notEOF))
{
  read line(x)
  display line(x)
  x+5
}

I got to go now, I'll post a larger explanation in 30 min or so.

[Bobboau]

wouldn't a sort of
cout << fstream;
work, I'm not sure honestly, you just want to display the contents of a file on screen?

[aldo_14]

Um... I think this Java code *should* work - or be close-enough to it

Code: [Select]


public class HackyThing  {

public static void main(String[] args)  {
try{
String filename = args[0];
}
catch(ArrayIndexOutOfBoundException e)  {
System.out.println("Dodgy args!");
System.exit(0);
}


File in = new File(filename);
BufferedReader br = new BufferedReader(new FileReader(in));
try{
while(true){
//get and print
System.out.println(br.readLine());
//skip 5
br.readline();
br.readline();
br.readline();
br.readline();
br.readline();
}
}
}
//thrown when no more lines to read? (need to check)
catch(IOException e) {
br.close();
//do nothing(?)
}
}


Originally posted by Bobboau
wouldn't a sort of
cout << fstream;
work, I'm not sure honestly, you just want to display the contents of a file on screen?

Every fifth line of a file, I think.  I might have some old C lying about to do this, now I think of it....

[kasperl]

Ok, here's the rest of the post:

I'm going to be helping someone with a rather dodgy net connection to get something more reliable then TCP/IP of pidgeon post. I wrote a batch file, which she's running each time the conn goes down.

Code: [Select]


ECHO "!TEST:!" >> netlog.txt

ECHO "%TIME%" >> netlog.txt
ECHO "%DATE%" >> netlog.txt

ECHO "DNS:" >> netlog.txt
ping {{{{{DNSIP}}}}} >> netlog.txt
ECHO "!EINDE:DNS!" >> netlog.txt

ECHO "GATE:"
ping {{{gatewayIP}}}>> netlog.txt
ECHO "!EINDE:GATE!" >> netlog.txt

ECHO "LCL:" >> netlog.txt
ping [url]www.lcl.nl[/url] >> netlog.txt
ECHO "!EINDE:LCL!" >> netlog.txt
ECHO "!EINDE:TEST!" >> netlog.txt

(Einde=end in Dutch, but I wasn't going to fool around using possibly restrictied words in a langauge I don't know all that well.)


The thing is, I want to parse the lot of it, to produce some statistics on what's going on. Im pinging the DNS, the gateway, and some random site to see what's working and what's not. I want to have certain lines in a var, so I can do comparisons and counting with that var. For example, I'm logging the time and date, and want those so I can get a outages/hour thingy. I also want to get the ping statistics into a var, so I can tear that apart to get all the numbers to add to the stats.

I tried a C++ implementation, but C++ For Dummies is about as understandable as an MS help file.

I can simply count line numbers to get the right data into the right var, but I can't read from a specified line in C, or at least I haven't found the function that lets me.

[aldo_14]

Aaah..... so you want to parse the log file, yes?

[kasperl]

Yes.

This thing is going to be over 10kb, most likely, I'd rather not do all the parsing with the Eyeball program by Human Interface Solutions. It's rather buggy after prolonged use.

In QuickBasic, I might be able to hack this nice and quick, but QB won't compile under XP....

[aldo_14]

Um... gimme a sample file (small, but representative), and a couple of hours to finish work & have dinner, and I think I could knock something up very quickly - albeit in java (although should be able to jar-package it so it's nicely runnable) - for you.

[kasperl]

I love you.

http://www.huiswerksite.nl/kasperl/netlog.txt

And a quick translation, in case you care:

Code: [Select]


"!TEST:!" //starting this block
"21:45:26,41"  //on this time
"za 26-03-2005"  // and this date
"DNS:"  //starting the DNS testing block


Pingen naar 127.0.0.1 met 32 byte gegevens: //pinging to blahblah



Antwoord van 127.0.0.1: bytes=32 tijd<1 ms TTL=128 //answer
(3x the above)


Ping-statistieken voor 127.0.0.1: //ping stats

    Pakketten: verzonden = 4, ontvangen = 4, verloren = 0 //send, recieved, lost

    (0% verlies).De gemiddelde tijd voor het uitvoeren van ‚‚n  bewerking in milliseconden: //precentage loss

    Minimum = 0ms, Maximum = 0ms, Gemiddelde = 0ms //min/max/avg times in ms

"!EINDE:DNS!" //end of the DNS block
 //here the gateway block begins, but I forgot to echo a start for that, sorry
//this bit is the exact same as the above.
"!EINDE:GATE!"  // you get the idea
"LCL:"  //starting the block testing my school's website.

//ditto as the two above
"!EINDE:LCL!"  //ending the site testing block
"!EINDE:TEST!"  //ending the block



[aldo_14]

Eeep....dutch.  This should be..interesting

Ok, what do you need to count - the 'Ping-statistieken voor 127.0.0.1:' bits?  

(so.. do you want summed / average totals for all the fields under there?)

[kasperl]

Mainly I want the ping times in some kind of stats (i.e. the all time maximums, and an all time average calculated), average package loss, and how often the package loss is 4 out of 4. This all for all 3 of the tests I'm running each time.

If you get me some kind of working code for getting the above, I'd already be a very happy man. I'll google some freeware java compiler, so I can always hack some more into it if I think I need it. The file parsing, and a few basic operations should be enough to allow me to copy/paste/hack in the rest

Thanks again, I really appreciate this.

[aldo_14]

Okey-doke.

NB: you can get the java SDK with compiler (Java has a bunch built in libraries, unlike C/C++) off the Sun website.  It's quite big, though, because of said libraries.

[aldo_14]

One thing; how are you calculating the percentage lost?

[aldo_14]

Code: [Select]


/*
 * Created on 28-Mar-2005
 *
 * Confidential Licensed Materials of Enigmatec corporation
 * Copyright Enigmatec Corporation @ 2005
 */
package dnetParse;

import java.io.BufferedReader;
import java.io.File;
import java.io.FileReader;
import java.io.IOException;
import java.io.StreamTokenizer;
import java.util.StringTokenizer;

import javax.swing.JOptionPane;

/**
 * The Driver class
 *
 * @author Alan White
 * @version $Revision: $
 */
/**
 * @author Alan White
 *
 *  Parses through a formatted text file and outputs the result
 */
public class Driver {
   
    /*
"!TEST:!" //starting this block
"21:45:26,41"  //on this time
"za 26-03-2005"  // and this date
"DNS:"  //starting the DNS testing block


Pingen naar 127.0.0.1 met 32 byte gegevens: //pinging to blahblah



Antwoord van 127.0.0.1: bytes=32 tijd<1 ms TTL=128 //answer
(3x the above)


Ping-statistieken voor 127.0.0.1: //ping stats

    Pakketten: verzonden = 4, ontvangen = 4, verloren = 0 //send, recieved, lost

    (0% verlies).De gemiddelde tijd voor het uitvoeren van ,, n  bewerking in milliseconden: //precentage loss

    Minimum = 0ms, Maximum = 0ms, Gemiddelde = 0ms //min/max/avg times in ms

"!EINDENS!" //end of the DNS block
 //here the gateway block begins, but I forgot to echo a start for that, sorry
//this bit is the exact same as the above.
"!EINDE:GATE!"  // you get the idea
"LCL:"  //starting the block testing my school's website.

//ditto as the two above
"!EINDE:LCL!"  //ending the site testing block
"!EINDE:TEST!"  //ending the block
     */
   
    public static final boolean VERBOSE_OUTPUT = true;
   
    /** start looking at content after this string */
    public static final String PACKET_HEADER = "Pakketten:";
    /** lost packets*/
    public static final String SENT_PACKETS = "verzonden";
    /** lost packets*/
    public static final String RECEIVED_PACKETS = "ontvangen";
    /** lost packets*/
    public static final String LOST_PACKETS = "verloren";
    /** Minimum; just look for this string and get the int afterwards!*/
     public static final String MIN = "Minimum";
     /** Maximum; just look for this string and get the int afterwards!*/
     public static final String MAX = "Maximum";
     /** Average.  In dutch.  See above.. */
     public static final String AVG = "Gemiddelde";
     /** ms appended to min & max vals - strip this to get the int */
     public static final String MIN_MAX_UNIT="ms";
     
     //////////////////////////////////////////////////////running totals/////////////////////////////////
     /** number of min/max/avg readings made */
     private int pingIterations =0;    
     /** number of packet send/receive/lost readings mage */
     private int packetIterations =0;
     /** summed total of min */
     private float minTotal = 0;    
     /** summed total of max  */
     private float maxTotal =0;
     /** summed total of avg  */
     private float avgTotal =0;
     /** number of times all packets are lost */
     private int allLost = 0;
     /** percentage total */
     private long percentLost = 0;
     /** total received */
     private int totalReceived=0;
     /** total sent*/
     private int totalSent = 0;
     /** total lost*/
     private int totalLost = 0;
     /** ms time at start */
     private long start;
     /** ms time at end */
     private long end;
     /** reads stuff */
     private BufferedReader reader;
    /**
     * Runs it (duh)
     * @param args String array, entry is the filename (assume is located in same dir)
     * @throws Exception if the Driver constructor screws up.  Sorry, couldn't be bothered handling it......
     */
    public static void main(String[] args) throws Exception {
        if(args.length<1)  {
            System.out.println("Not enough args - need at least one!");
            System.exit(0);
        }
        Driver d = new Driver(args[0]);
        d.trim();
        d.parse();
        d.result();
    }
   
    /**
     * Create a new Driver object, encapsulate misc stuff in here
     * @param filename
     */
    public Driver(String filename)  throws Exception{
        reader = new BufferedReader(new FileReader(new File(filename)));
        start = System.currentTimeMillis();
    }
   
    /**
     * Removes any crap from the buffered file; stub method at present
     */
    public void trim()    {
        //TODO any trimming required?
        out("Trim done");
    }
   
    /**
     * Does the dirty work of actually parsing; horrible linear parse method
     */
    public void parse()  {
        out("Beginning parse");
        String line;
        try  {
        while( (line=reader.readLine())!=null)  {
            out("Read line; " + line);
            handleLine(line);
        }
        }
        catch(IOException e)  {
            System.out.println("Error reading file; " + e.toString());
            //don't abort program, let the result display what there is...
        }
        catch(Exception e)  {
            System.out.println("Error in parse " + e.toString());
            System.exit(0);
        }
        out("Completed parse");
    }
   
    /**
     * Handles the parsing of indiviudal lines
     * @param line
     */
    protected void handleLine(String line)  throws Exception {
        out("Handling line; " + line);
        line = line.trim(); //remove start & end WS
       
        //potential for template method here...
        if(line.startsWith(MIN))  {
            //break up and sort out
            handlePingLine(line);
        }
        else if(line.startsWith(PACKET_HEADER))  {
            //find lost stat
            handlePacketLine(line);
        }
        else  {
            //no handling....
            out("No handling needed");
        }
    }
   
    /**
     * Handles a ping line; in form ala Minimum = 11ms, Maximum = 12ms, Gemiddelde = 11ms
     *
     * Fairly strict linear parse; expects EXACT formatting
     * @param line
     */
    protected void handlePingLine(String line)  throws Exception {
        out("Parsing Ping line");
        StringTokenizer st = new StringTokenizer(line);
        String current;
        //used to substring - kept here to only calc once
        int subLength = (MIN_MAX_UNIT+",").length();
       
        //min token
        if(!  ( current = st.nextToken()  ).equals(MIN))  {
            throw new Exception("Min not found! Got "+current+" instead");
            }
        else  {
            out("Got "+current);  
            }
        if(!  ( current = st.nextToken()  ).equals("="))  {
            throw new Exception("= not found! Got "+current+" instead");
            }
        else  {
            out("Got "+current);  
            }        
       
        //this will be the ##ms bit...
        current = st.nextToken();
        if(!current.endsWith(MIN_MAX_UNIT+","))  {
            throw new Exception("Unexpected ms format; got "+current+" instead");
        }
        else {
            current = current.substring(0, current.length()-subLength);
            int minMs = Integer.parseInt(current);
            minTotal += minMs;
            out("Got min of "+minMs+" -  total is now "+minTotal);
        }
       
        //max token
        if(!  ( current = st.nextToken()  ).equals(MAX))  {
            throw new Exception("Max not found! Got "+current+" instead");
            }
        else  {
            out("Got "+current);  
            }
        if(!  ( current = st.nextToken()  ).equals("="))  {
            throw new Exception("= not found! Got "+current+" instead");
            }
        else  {
            out("Got "+current);  
            }        
       
        //this will be the ##ms bit...
        current = st.nextToken();
        if(!current.endsWith(MIN_MAX_UNIT+","))  {
            throw new Exception("Unexpected ms format; got "+current+" instead");
        }
        else {
            current = current.substring(0, current.length()-subLength);
            int maxMs = Integer.parseInt(current);
            maxTotal += maxMs;
            out("Got max of "+maxMs+" -  total is now "+maxTotal);
        }      
       
        //Gemiddelde token
        if(!  ( current = st.nextToken()  ).equals(AVG))  {
            throw new Exception("Avg not found! Got "+current+" instead");
            }
        else  {
            out("Got "+current);  
            }
        if(!  ( current = st.nextToken()  ).equals("="))  {
            throw new Exception("= not found! Got "+current+" instead");
            }
        else  {
            out("Got "+current);  
            }        
       
        //this will be the ##ms bit...
        current = st.nextToken();
        if(!current.endsWith(MIN_MAX_UNIT))  {
            throw new Exception("Unexpected ms format; got "+current+" instead");
        }
        else {
            current = current.substring(0, current.length()-MIN_MAX_UNIT.length());
            int avgMs = Integer.parseInt(current);
            avgTotal += avgMs;
            out("Got avg of "+avgMs+" -  total is now "+avgTotal);
        }        
       
        out("Ping line parsed");
        pingIterations++;
    }
   
    /**
     * Handles packet line; in form ala Pakketten: verzonden = 4, ontvangen = 4, verloren = 0
     *
     * Fairly strict linear parse; expects EXACT formatting
     * @param line
     */
    protected void handlePacketLine(String line)  throws Exception {
        out("Parsing Packets line");
        StringTokenizer st = new StringTokenizer(line);
       String current;
       int received;
       int sent;
       int lost;
       
        //Pakketten
       if(!  ( current = st.nextToken()  ).equals(PACKET_HEADER))  {
           throw new Exception(PACKET_HEADER + " not found! Got "+current+" instead");
           }
       else  {
           out("Got "+current);  
           }
       
        //verzonden
       if(!  ( current = st.nextToken()  ).equals(SENT_PACKETS))  {
           throw new Exception(SENT_PACKETS+" not found! Got "+current+" instead");
           }
       else  {
           out("Got "+current);  
           }    
       
        //=
       if(!  ( current = st.nextToken()  ).equals("="))  {
           throw new Exception("= not found! Got "+current+" instead");
           }
       else  {
           out("Got "+current);  
           }    
       
        //number,
        if( !( current=st.nextToken() ).endsWith(",") )  {
            throw new Exception(", suffix not found! Got "+current+" instead");
        }
        else  {
            //get number
            current = current.substring(0, current.length()-1);
            sent = Integer.parseInt(current);
            out( sent + " packets sent");
        }
       
        //ontvangen
       if(!  ( current = st.nextToken()  ).equals(RECEIVED_PACKETS))  {
           throw new Exception(RECEIVED_PACKETS+" not found! Got "+current+" instead");
           }
       else  {
           out("Got "+current);  
           }    
       
        //=
       if(!  ( current = st.nextToken()  ).equals("="))  {
           throw new Exception("= not found! Got "+current+" instead");
           }
       else  {
           out("Got "+current);  
           }    
       
        //number,
       if( !( current=st.nextToken() ).endsWith(",") )  {
           throw new Exception(", suffix not found! Got "+current+" instead");
        }
       else  {
           //get number
           current = current.substring(0, current.length()-1);
           received = Integer.parseInt(current);
           out(received + " packets received");
       }
       
        //verloren
       if(!  ( current = st.nextToken()  ).equals(LOST_PACKETS))  {
           throw new Exception(LOST_PACKETS+" not found! Got "+current+" instead");
           }
       else  {
           out("Got "+current);  
           }    
       
        //=
       if(!  ( current = st.nextToken()  ).equals("="))  {
           throw new Exception("= not found! Got "+current+" instead");
           }
       else  {
           out("Got "+current);  
           }    
       
        //number
       current=st.nextToken();
       lost = Integer.parseInt(current);
       out(lost + " packets lost");
       
       //calculate percentage lost
       totalLost += lost;
       totalReceived += received;
       totalSent += received;
       
       //check if all lost
       float pcg = (lost/received)*100;
       out("Percentage loss = "+pcg);
       
       percentLost += pcg;
       
       if(pcg==100)  {
           out("All packets lost!");
           allLost++;
       }
       
        out("Packets line parsed");
        packetIterations++;
    }
   
    /**
     * Shows the result.  Whee!
     */
    public void result()  {
        //text
        end = System.currentTimeMillis();
        long time = end - start;
        System.out.println("====DONE====");
        System.out.println("Parsing operation took " + time + " ms");
        System.out.println();
        System.out.println("Number of ping readings; " + pingIterations);
        System.out.println("Number of packet readings; " + packetIterations);
        System.out.println();
        if (packetIterations>0){
            System.out.println("Times all packets were lost: " + allLost);
            System.out.println("Percentage loss: " + (  percentLost / packetIterations  )  );
            System.out.println("Average packet receipt; "+totalReceived+" over "+packetIterations+" its; " + (totalReceived/packetIterations));
            System.out.println("Average packet send; "+totalSent+" over "+packetIterations+" its; " + (totalSent/packetIterations));
            System.out.println("Average packet lost; "+totalLost+" over "+packetIterations+" its; " + (totalLost/packetIterations));
        }
           if  (pingIterations>0) {
            System.out.println("Max ("+maxTotal+") averaged over "+pingIterations+" its "+(  maxTotal/pingIterations  )  );
            System.out.println("Min ("+minTotal+") averaged over "+pingIterations+" its "+(  minTotal/pingIterations  )  );
            System.out.println("Average ("+avgTotal+") over "+pingIterations+" its "+(  avgTotal/pingIterations  )  );
        }
        System.out.println("====END====");
    }
   
    /**
     * Used for debug output; only displays stuff if VERBOSE is true
     * @param msg String message
     */
    private void out(String msg)  {
        if(VERBOSE_OUTPUT)  {
            System.out.println(System.currentTimeMillis() + ">>>" + msg);
        }
    }
}


Uploading the jar in a sec; not sure on robustness, % loss calculation may be flaky (it was easier to recalculate than parse)

EDIT;

Here goes.  

www.aldo14.f2s.com/downloadable_stuff/dutch_parse_thing.zip

EDIT2; oh... having VERBOSE_OUTPUT set to false will massively reduce parse time (by 10 times or so); I probably should have turned that off......

[kasperl]

Sweet! You even did this in a way I can reconfigure the lot of it to work with an English windoze if I need to.

And it actually looks like Java is a bit easier than C++, especially looking at how the file handling is done. Would you agree that for quick&dirty hacks Java>C++?


I'm not sure where you handle the distinction between the pings to the DNS, the gateway and the website, but I might be able to do that myself. I'm not going to be working on this tonight, though.


(I've already had a tad more wine than I should've, and I've got Oceans' Eleven on one net, and Star Trek and Stargate on the other, so if you'll pardon me..)

Thanks again, I really like this.

[aldo_14]

Originally posted by kasperl
Sweet! You even did this in a way I can reconfigure the lot of it to work with an English windoze if I need to.

And it actually looks like Java is a bit easier than C++, especially looking at how the file handling is done. Would you agree that for quick&dirty hacks Java>C++?

IMO, yes.  But I'm not a user of C++, after all.

Java has a lot more inbuilt libraries than C++ IIRC, which helps a lot.


I'm not sure where you handle the distinction between the pings to the DNS, the gateway and the website, but I might be able to do that myself. I'm not going to be working on this tonight, though.


Ah... to be honest, I never noticed that bit.  ****.

Hmm.... I'll  have a fiddle tonight.  I might be able to split it up pretty simple by implementing something in 'trim' to break up the file, and modifying the arguement into parse....

If you add in an echo for the start of GATE, I should be able sort it pretty simply.  I may just break up the file or something, there's a few ways to solve it.

Balls.  Annoyed I missed that, now.  Especially as I could have fitted in some recursion or maybe a tree storage structure.


(I've already had a tad more wine than I should've, and I've got Oceans' Eleven on one net, and Star Trek and Stargate on the other, so if you'll pardon me..)

Thanks again, I really like this.

[aldo_14]

Ok, had a think.  No need to change the 'Driver' code; just add an echo for the GATE start, and I'll use that to create a class that simply breaks up the original file into sections, and then uses the Driver functionality to parse those 3 individual files.

Should be fairly simple... it's a little hacky, granted.  On the other hand, I *should* be able to structure it so it will be able to identify an aribtrary amount of tags (i.e. above 'gate', 'dns', etc), so it'll be more flexible in that way.

[Sandwich]

Holy crapola, Aldoman! Tell me that that whole balagan of code isn't just to output every 5th line of a text file, cuz you could do that in about 10 lines of PHP.

[aldo_14]

Originally posted by Sandwich
Holy crapola, Aldoman! Tell me that that whole balagan of code isn't just to output every 5th line of a text file, cuz you could do that in about 10 lines of PHP.

No, it's to parse in the data from it and calculate the summations & averages.  The bulk of it is, admittedly, to enforce a very tight format check on the input.

It's actually an incredibly simple piece of code, though.

[WMCoolmon]

Originally posted by kasperl
Sweet! You even did this in a way I can reconfigure the lot of it to work with an English windoze if I need to.

And it actually looks like Java is a bit easier than C++, especially looking at how the file handling is done. Would you agree that for quick&dirty hacks Java>C++?

Having done a little Java learning (and taking a look at the parser code) you could do it with about the same amount of trouble, assuming you had a good parsing library on hand. There probably wouldn't be a big difference in speed, either.

If you want quick and dirty for file parsing, you go to C or PHP. What aldo wrote definitely wouldn't classify as a 'quick and dirty hack'.

[aldo_14]

No, it's a quick and dirty hack with exceptions........there are far better ways to do this sort of thing with recursion or tree-building or regular expressions, or really anything beyond a strict-format linear run.  But they take longer

Anyways, kasp; give me that start echo and I can rejig it very quickly to handle the different bits.