Hard Light Productions Forums

Off-Topic Discussion => General Discussion => Topic started by: kasperl on March 28, 2005, 08:42:44 am

Title: Parsing a text file
Post by: kasperl on March 28, 2005, 08:42:44 am
Ok, I need a quick way of doing this bit of pseudo code in C, C++, PHP, or whatever you can think of that works under XP and has a free compiler.

while(file(notEOF))
{
  read line(x)
  display line(x)
  x+5
}

I got to go now, I'll post a larger explanation in 30 min or so.
Title: Parsing a text file
Post by: Bobboau on March 28, 2005, 08:47:03 am
wouldn't a sort of
cout << fstream;
work, I'm not sure honestly, you just want to display the contents of a file on screen?
Title: Parsing a text file
Post by: aldo_14 on March 28, 2005, 08:52:46 am
Um... I think this Java code *should* work - or be close-enough to it

Code: [Select]

public class HackyThing  {

public static void main(String[] args)  {
try{
String filename = args[0];
}
catch(ArrayIndexOutOfBoundException e)  {
System.out.println("Dodgy args!");
System.exit(0);
}


File in = new File(filename);
BufferedReader br = new BufferedReader(new FileReader(in));
try{
while(true){
//get and print
System.out.println(br.readLine());
//skip 5
br.readline();
br.readline();
br.readline();
br.readline();
br.readline();
}
}
}
//thrown when no more lines to read? (need to check)
catch(IOException e) {
br.close();
//do nothing(?)
}
}


Quote
Originally posted by Bobboau
wouldn't a sort of
cout << fstream;
work, I'm not sure honestly, you just want to display the contents of a file on screen?


Every fifth line of a file, I think.  I might have some old C lying about to do this, now I think of it....
Title: Parsing a text file
Post by: kasperl on March 28, 2005, 09:27:58 am
Ok, here's the rest of the post:

I'm going to be helping someone with a rather dodgy net connection to get something more reliable then TCP/IP of pidgeon post. I wrote a batch file, which she's running each time the conn goes down.
Code: [Select]

ECHO "!TEST:!" >> netlog.txt

ECHO "%TIME%" >> netlog.txt
ECHO "%DATE%" >> netlog.txt

ECHO "DNS:" >> netlog.txt
ping {{{{{DNSIP}}}}} >> netlog.txt
ECHO "!EINDE:DNS!" >> netlog.txt

ECHO "GATE:"
ping {{{gatewayIP}}}>> netlog.txt
ECHO "!EINDE:GATE!" >> netlog.txt

ECHO "LCL:" >> netlog.txt
ping [url]www.lcl.nl[/url] >> netlog.txt
ECHO "!EINDE:LCL!" >> netlog.txt
ECHO "!EINDE:TEST!" >> netlog.txt

(Einde=end in Dutch, but I wasn't going to fool around using possibly restrictied words in a langauge I don't know all that well.)


The thing is, I want to parse the lot of it, to produce some statistics on what's going on. Im pinging the DNS, the gateway, and some random site to see what's working and what's not. I want to have certain lines in a var, so I can do comparisons and counting with that var. For example, I'm logging the time and date, and want those so I can get a outages/hour thingy. I also want to get the ping statistics into a var, so I can tear that apart to get all the numbers to add to the stats.

I tried a C++ implementation, but C++ For Dummies is about as understandable as an MS help file.

I can simply count line numbers to get the right data into the right var, but I can't read from a specified line in C, or at least I haven't found the function that lets me.
Title: Parsing a text file
Post by: aldo_14 on March 28, 2005, 09:43:07 am
Aaah..... so you want to parse the log file, yes?
Title: Parsing a text file
Post by: kasperl on March 28, 2005, 09:43:56 am
Yes.

This thing is going to be over 10kb, most likely, I'd rather not do all the parsing with the Eyeball program by Human Interface Solutions. It's rather buggy after prolonged use.

In QuickBasic, I might be able to hack this nice and quick, but QB won't compile under XP....
Title: Parsing a text file
Post by: aldo_14 on March 28, 2005, 09:47:54 am
Um... gimme a sample file (small, but representative), and a couple of hours to finish work & have dinner, and I think I could knock something up very quickly - albeit in java (although should be able to jar-package it so it's nicely runnable) - for you.
Title: Parsing a text file
Post by: kasperl on March 28, 2005, 09:54:27 am
I love you.

http://www.huiswerksite.nl/kasperl/netlog.txt

And a quick translation, in case you care:
Code: [Select]

"!TEST:!" //starting this block
"21:45:26,41"  //on this time
"za 26-03-2005"  // and this date
"DNS:"  //starting the DNS testing block


Pingen naar 127.0.0.1 met 32 byte gegevens: //pinging to blahblah



Antwoord van 127.0.0.1: bytes=32 tijd<1 ms TTL=128 //answer
(3x the above)


Ping-statistieken voor 127.0.0.1: //ping stats

    Pakketten: verzonden = 4, ontvangen = 4, verloren = 0 //send, recieved, lost

    (0% verlies).De gemiddelde tijd voor het uitvoeren van ‚‚n  bewerking in milliseconden: //precentage loss

    Minimum = 0ms, Maximum = 0ms, Gemiddelde = 0ms //min/max/avg times in ms

"!EINDE:DNS!" //end of the DNS block
 //here the gateway block begins, but I forgot to echo a start for that, sorry
//this bit is the exact same as the above.
"!EINDE:GATE!"  // you get the idea
"LCL:"  //starting the block testing my school's website.

//ditto as the two above
"!EINDE:LCL!"  //ending the site testing block
"!EINDE:TEST!"  //ending the block

Title: Parsing a text file
Post by: aldo_14 on March 28, 2005, 10:00:14 am
Eeep....dutch.  This should be..interesting :D

Ok, what do you need to count - the 'Ping-statistieken voor 127.0.0.1:' bits?  

(so.. do you want summed / average totals for all the fields under there?)
Title: Parsing a text file
Post by: kasperl on March 28, 2005, 10:04:23 am
Mainly I want the ping times in some kind of stats (i.e. the all time maximums, and an all time average calculated), average package loss, and how often the package loss is 4 out of 4. This all for all 3 of the tests I'm running each time.

If you get me some kind of working code for getting the above, I'd already be a very happy man. I'll google some freeware java compiler, so I can always hack some more into it if I think I need it. The file parsing, and a few basic operations should be enough to allow me to copy/paste/hack in the rest

Thanks again, I really appreciate this.
Title: Parsing a text file
Post by: aldo_14 on March 28, 2005, 10:10:21 am
Okey-doke.

NB: you can get the java SDK with compiler (Java has a bunch built in libraries, unlike C/C++) off the Sun website.  It's quite big, though, because of said libraries.
Title: Parsing a text file
Post by: aldo_14 on March 28, 2005, 12:04:18 pm
One thing; how are you calculating the percentage lost?
Title: Parsing a text file
Post by: aldo_14 on March 28, 2005, 12:13:27 pm
Code: [Select]

/*
 * Created on 28-Mar-2005
 *
 * Confidential Licensed Materials of Enigmatec corporation
 * Copyright Enigmatec Corporation @ 2005
 */
package dnetParse;

import java.io.BufferedReader;
import java.io.File;
import java.io.FileReader;
import java.io.IOException;
import java.io.StreamTokenizer;
import java.util.StringTokenizer;

import javax.swing.JOptionPane;

/**
 * The Driver class
 *
 * @author Alan White
 * @version $Revision: $
 */
/**
 * @author Alan White
 *
 *  Parses through a formatted text file and outputs the result
 */
public class Driver {
   
    /*
"!TEST:!" //starting this block
"21:45:26,41"  //on this time
"za 26-03-2005"  // and this date
"DNS:"  //starting the DNS testing block


Pingen naar 127.0.0.1 met 32 byte gegevens: //pinging to blahblah



Antwoord van 127.0.0.1: bytes=32 tijd<1 ms TTL=128 //answer
(3x the above)


Ping-statistieken voor 127.0.0.1: //ping stats

    Pakketten: verzonden = 4, ontvangen = 4, verloren = 0 //send, recieved, lost

    (0% verlies).De gemiddelde tijd voor het uitvoeren van ,, n  bewerking in milliseconden: //precentage loss

    Minimum = 0ms, Maximum = 0ms, Gemiddelde = 0ms //min/max/avg times in ms

"!EINDENS!" //end of the DNS block
 //here the gateway block begins, but I forgot to echo a start for that, sorry
//this bit is the exact same as the above.
"!EINDE:GATE!"  // you get the idea
"LCL:"  //starting the block testing my school's website.

//ditto as the two above
"!EINDE:LCL!"  //ending the site testing block
"!EINDE:TEST!"  //ending the block
     */
   
    public static final boolean VERBOSE_OUTPUT = true;
   
    /** start looking at content after this string */
    public static final String PACKET_HEADER = "Pakketten:";
    /** lost packets*/
    public static final String SENT_PACKETS = "verzonden";
    /** lost packets*/
    public static final String RECEIVED_PACKETS = "ontvangen";
    /** lost packets*/
    public static final String LOST_PACKETS = "verloren";
    /** Minimum; just look for this string and get the int afterwards!*/
     public static final String MIN = "Minimum";
     /** Maximum; just look for this string and get the int afterwards!*/
     public static final String MAX = "Maximum";
     /** Average.  In dutch.  See above.. */
     public static final String AVG = "Gemiddelde";
     /** ms appended to min & max vals - strip this to get the int */
     public static final String MIN_MAX_UNIT="ms";
     
     //////////////////////////////////////////////////////running totals/////////////////////////////////
     /** number of min/max/avg readings made */
     private int pingIterations =0;    
     /** number of packet send/receive/lost readings mage */
     private int packetIterations =0;
     /** summed total of min */
     private float minTotal = 0;    
     /** summed total of max  */
     private float maxTotal =0;
     /** summed total of avg  */
     private float avgTotal =0;
     /** number of times all packets are lost */
     private int allLost = 0;
     /** percentage total */
     private long percentLost = 0;
     /** total received */
     private int totalReceived=0;
     /** total sent*/
     private int totalSent = 0;
     /** total lost*/
     private int totalLost = 0;
     /** ms time at start */
     private long start;
     /** ms time at end */
     private long end;
     /** reads stuff */
     private BufferedReader reader;
    /**
     * Runs it (duh)
     * @param args String array, entry is the filename (assume is located in same dir)
     * @throws Exception if the Driver constructor screws up.  Sorry, couldn't be bothered handling it......
     */
    public static void main(String[] args) throws Exception {
        if(args.length<1)  {
            System.out.println("Not enough args - need at least one!");
            System.exit(0);
        }
        Driver d = new Driver(args[0]);
        d.trim();
        d.parse();
        d.result();
    }
   
    /**
     * Create a new Driver object, encapsulate misc stuff in here
     * @param filename
     */
    public Driver(String filename)  throws Exception{
        reader = new BufferedReader(new FileReader(new File(filename)));
        start = System.currentTimeMillis();
    }
   
    /**
     * Removes any crap from the buffered file; stub method at present
     */
    public void trim()    {
        //TODO any trimming required?
        out("Trim done");
    }
   
    /**
     * Does the dirty work of actually parsing; horrible linear parse method
     */
    public void parse()  {
        out("Beginning parse");
        String line;
        try  {
        while( (line=reader.readLine())!=null)  {
            out("Read line; " + line);
            handleLine(line);
        }
        }
        catch(IOException e)  {
            System.out.println("Error reading file; " + e.toString());
            //don't abort program, let the result display what there is...
        }
        catch(Exception e)  {
            System.out.println("Error in parse " + e.toString());
            System.exit(0);
        }
        out("Completed parse");
    }
   
    /**
     * Handles the parsing of indiviudal lines
     * @param line
     */
    protected void handleLine(String line)  throws Exception {
        out("Handling line; " + line);
        line = line.trim(); //remove start & end WS
       
        //potential for template method here...
        if(line.startsWith(MIN))  {
            //break up and sort out
            handlePingLine(line);
        }
        else if(line.startsWith(PACKET_HEADER))  {
            //find lost stat
            handlePacketLine(line);
        }
        else  {
            //no handling....
            out("No handling needed");
        }
    }
   
    /**
     * Handles a ping line; in form ala Minimum = 11ms, Maximum = 12ms, Gemiddelde = 11ms
     *
     * Fairly strict linear parse; expects EXACT formatting
     * @param line
     */
    protected void handlePingLine(String line)  throws Exception {
        out("Parsing Ping line");
        StringTokenizer st = new StringTokenizer(line);
        String current;
        //used to substring - kept here to only calc once
        int subLength = (MIN_MAX_UNIT+",").length();
       
        //min token
        if(!  ( current = st.nextToken()  ).equals(MIN))  {
            throw new Exception("Min not found! Got "+current+" instead");
            }
        else  {
            out("Got "+current);  
            }
        if(!  ( current = st.nextToken()  ).equals("="))  {
            throw new Exception("= not found! Got "+current+" instead");
            }
        else  {
            out("Got "+current);  
            }        
       
        //this will be the ##ms bit...
        current = st.nextToken();
        if(!current.endsWith(MIN_MAX_UNIT+","))  {
            throw new Exception("Unexpected ms format; got "+current+" instead");
        }
        else {
            current = current.substring(0, current.length()-subLength);
            int minMs = Integer.parseInt(current);
            minTotal += minMs;
            out("Got min of "+minMs+" -  total is now "+minTotal);
        }
       
        //max token
        if(!  ( current = st.nextToken()  ).equals(MAX))  {
            throw new Exception("Max not found! Got "+current+" instead");
            }
        else  {
            out("Got "+current);  
            }
        if(!  ( current = st.nextToken()  ).equals("="))  {
            throw new Exception("= not found! Got "+current+" instead");
            }
        else  {
            out("Got "+current);  
            }        
       
        //this will be the ##ms bit...
        current = st.nextToken();
        if(!current.endsWith(MIN_MAX_UNIT+","))  {
            throw new Exception("Unexpected ms format; got "+current+" instead");
        }
        else {
            current = current.substring(0, current.length()-subLength);
            int maxMs = Integer.parseInt(current);
            maxTotal += maxMs;
            out("Got max of "+maxMs+" -  total is now "+maxTotal);
        }      
       
        //Gemiddelde token
        if(!  ( current = st.nextToken()  ).equals(AVG))  {
            throw new Exception("Avg not found! Got "+current+" instead");
            }
        else  {
            out("Got "+current);  
            }
        if(!  ( current = st.nextToken()  ).equals("="))  {
            throw new Exception("= not found! Got "+current+" instead");
            }
        else  {
            out("Got "+current);  
            }        
       
        //this will be the ##ms bit...
        current = st.nextToken();
        if(!current.endsWith(MIN_MAX_UNIT))  {
            throw new Exception("Unexpected ms format; got "+current+" instead");
        }
        else {
            current = current.substring(0, current.length()-MIN_MAX_UNIT.length());
            int avgMs = Integer.parseInt(current);
            avgTotal += avgMs;
            out("Got avg of "+avgMs+" -  total is now "+avgTotal);
        }        
       
        out("Ping line parsed");
        pingIterations++;
    }
   
    /**
     * Handles packet line; in form ala Pakketten: verzonden = 4, ontvangen = 4, verloren = 0
     *
     * Fairly strict linear parse; expects EXACT formatting
     * @param line
     */
    protected void handlePacketLine(String line)  throws Exception {
        out("Parsing Packets line");
        StringTokenizer st = new StringTokenizer(line);
       String current;
       int received;
       int sent;
       int lost;
       
        //Pakketten
       if(!  ( current = st.nextToken()  ).equals(PACKET_HEADER))  {
           throw new Exception(PACKET_HEADER + " not found! Got "+current+" instead");
           }
       else  {
           out("Got "+current);  
           }
       
        //verzonden
       if(!  ( current = st.nextToken()  ).equals(SENT_PACKETS))  {
           throw new Exception(SENT_PACKETS+" not found! Got "+current+" instead");
           }
       else  {
           out("Got "+current);  
           }    
       
        //=
       if(!  ( current = st.nextToken()  ).equals("="))  {
           throw new Exception("= not found! Got "+current+" instead");
           }
       else  {
           out("Got "+current);  
           }    
       
        //number,
        if( !( current=st.nextToken() ).endsWith(",") )  {
            throw new Exception(", suffix not found! Got "+current+" instead");
        }
        else  {
            //get number
            current = current.substring(0, current.length()-1);
            sent = Integer.parseInt(current);
            out( sent + " packets sent");
        }
       
        //ontvangen
       if(!  ( current = st.nextToken()  ).equals(RECEIVED_PACKETS))  {
           throw new Exception(RECEIVED_PACKETS+" not found! Got "+current+" instead");
           }
       else  {
           out("Got "+current);  
           }    
       
        //=
       if(!  ( current = st.nextToken()  ).equals("="))  {
           throw new Exception("= not found! Got "+current+" instead");
           }
       else  {
           out("Got "+current);  
           }    
       
        //number,
       if( !( current=st.nextToken() ).endsWith(",") )  {
           throw new Exception(", suffix not found! Got "+current+" instead");
        }
       else  {
           //get number
           current = current.substring(0, current.length()-1);
           received = Integer.parseInt(current);
           out(received + " packets received");
       }
       
        //verloren
       if(!  ( current = st.nextToken()  ).equals(LOST_PACKETS))  {
           throw new Exception(LOST_PACKETS+" not found! Got "+current+" instead");
           }
       else  {
           out("Got "+current);  
           }    
       
        //=
       if(!  ( current = st.nextToken()  ).equals("="))  {
           throw new Exception("= not found! Got "+current+" instead");
           }
       else  {
           out("Got "+current);  
           }    
       
        //number
       current=st.nextToken();
       lost = Integer.parseInt(current);
       out(lost + " packets lost");
       
       //calculate percentage lost
       totalLost += lost;
       totalReceived += received;
       totalSent += received;
       
       //check if all lost
       float pcg = (lost/received)*100;
       out("Percentage loss = "+pcg);
       
       percentLost += pcg;
       
       if(pcg==100)  {
           out("All packets lost!");
           allLost++;
       }
       
        out("Packets line parsed");
        packetIterations++;
    }
   
    /**
     * Shows the result.  Whee!
     */
    public void result()  {
        //text
        end = System.currentTimeMillis();
        long time = end - start;
        System.out.println("====DONE====");
        System.out.println("Parsing operation took " + time + " ms");
        System.out.println();
        System.out.println("Number of ping readings; " + pingIterations);
        System.out.println("Number of packet readings; " + packetIterations);
        System.out.println();
        if (packetIterations>0){
            System.out.println("Times all packets were lost: " + allLost);
            System.out.println("Percentage loss: " + (  percentLost / packetIterations  )  );
            System.out.println("Average packet receipt; "+totalReceived+" over "+packetIterations+" its; " + (totalReceived/packetIterations));
            System.out.println("Average packet send; "+totalSent+" over "+packetIterations+" its; " + (totalSent/packetIterations));
            System.out.println("Average packet lost; "+totalLost+" over "+packetIterations+" its; " + (totalLost/packetIterations));
        }
           if  (pingIterations>0) {
            System.out.println("Max ("+maxTotal+") averaged over "+pingIterations+" its "+(  maxTotal/pingIterations  )  );
            System.out.println("Min ("+minTotal+") averaged over "+pingIterations+" its "+(  minTotal/pingIterations  )  );
            System.out.println("Average ("+avgTotal+") over "+pingIterations+" its "+(  avgTotal/pingIterations  )  );
        }
        System.out.println("====END====");
    }
   
    /**
     * Used for debug output; only displays stuff if VERBOSE is true
     * @param msg String message
     */
    private void out(String msg)  {
        if(VERBOSE_OUTPUT)  {
            System.out.println(System.currentTimeMillis() + ">>>" + msg);
        }
    }
}


Uploading the jar in a sec; not sure on robustness, % loss calculation may be flaky (it was easier to recalculate than parse)

EDIT;

Here goes.  

www.aldo14.f2s.com/downloadable_stuff/dutch_parse_thing.zip

EDIT2; oh... having VERBOSE_OUTPUT set to false will massively reduce parse time (by 10 times or so); I probably should have turned that off......
Title: Parsing a text file
Post by: kasperl on March 28, 2005, 12:34:32 pm
Sweet! You even did this in a way I can reconfigure the lot of it to work with an English windoze if I need to.

And it actually looks like Java is a bit easier than C++, especially looking at how the file handling is done. Would you agree that for quick&dirty hacks Java>C++?


I'm not sure where you handle the distinction between the pings to the DNS, the gateway and the website, but I might be able to do that myself. I'm not going to be working on this tonight, though.


(I've already had a tad more wine than I should've, and I've got Oceans' Eleven on one net, and Star Trek and Stargate on the other, so if you'll pardon me..)

Thanks again, I really like this.
Title: Parsing a text file
Post by: aldo_14 on March 28, 2005, 12:50:19 pm
Quote
Originally posted by kasperl
Sweet! You even did this in a way I can reconfigure the lot of it to work with an English windoze if I need to.

And it actually looks like Java is a bit easier than C++, especially looking at how the file handling is done. Would you agree that for quick&dirty hacks Java>C++?

IMO, yes.  But I'm not a user of C++, after all.

Java has a lot more inbuilt libraries than C++ IIRC, which helps a lot.


I'm not sure where you handle the distinction between the pings to the DNS, the gateway and the website, but I might be able to do that myself. I'm not going to be working on this tonight, though.


Ah... to be honest, I never noticed that bit.  ****.

Hmm.... I'll  have a fiddle tonight.  I might be able to split it up pretty simple by implementing something in 'trim' to break up the file, and modifying the arguement into parse....

If you add in an echo for the start of GATE, I should be able sort it pretty simply.  I may just break up the file or something, there's a few ways to solve it.

Balls.  Annoyed I missed that, now.  Especially as I could have fitted in some recursion or maybe a tree storage structure.


(I've already had a tad more wine than I should've, and I've got Oceans' Eleven on one net, and Star Trek and Stargate on the other, so if you'll pardon me..)

Thanks again, I really like this.
Title: Parsing a text file
Post by: aldo_14 on March 28, 2005, 03:48:29 pm
Ok, had a think.  No need to change the 'Driver' code; just add an echo for the GATE start, and I'll use that to create a class that simply breaks up the original file into sections, and then uses the Driver functionality to parse those 3 individual files.

Should be fairly simple... it's a little hacky, granted.  On the other hand, I *should* be able to structure it so it will be able to identify an aribtrary amount of tags (i.e. above 'gate', 'dns', etc), so it'll be more flexible in that way.
Title: Parsing a text file
Post by: Sandwich on March 28, 2005, 04:30:50 pm
Holy crapola, Aldoman! Tell me that that whole balagan of code isn't just to output every 5th line of a text file, cuz you could do that in about 10 lines of PHP. :wtf:
Title: Parsing a text file
Post by: aldo_14 on March 28, 2005, 04:35:49 pm
Quote
Originally posted by Sandwich
Holy crapola, Aldoman! Tell me that that whole balagan of code isn't just to output every 5th line of a text file, cuz you could do that in about 10 lines of PHP. :wtf:


No, it's to parse in the data from it and calculate the summations & averages.  The bulk of it is, admittedly, to enforce a very tight format check on the input.

It's actually an incredibly simple piece of code, though.
Title: Parsing a text file
Post by: WMCoolmon on March 28, 2005, 07:04:22 pm
Quote
Originally posted by kasperl
Sweet! You even did this in a way I can reconfigure the lot of it to work with an English windoze if I need to.

And it actually looks like Java is a bit easier than C++, especially looking at how the file handling is done. Would you agree that for quick&dirty hacks Java>C++?


Having done a little Java learning (and taking a look at the parser code) you could do it with about the same amount of trouble, assuming you had a good parsing library on hand. There probably wouldn't be a big difference in speed, either.

If you want quick and dirty for file parsing, you go to C or PHP. What aldo wrote definitely wouldn't classify as a 'quick and dirty hack'. ;)
Title: Parsing a text file
Post by: aldo_14 on March 29, 2005, 04:07:41 am
No, it's a quick and dirty hack with exceptions........there are far better ways to do this sort of thing with recursion or tree-building or regular expressions, or really anything beyond a strict-format linear run.  But they take longer :D

Anyways, kasp; give me that start echo and I can rejig it very quickly to handle the different bits.
Title: Parsing a text file
Post by: Grug on March 29, 2005, 04:14:05 am
All hail Java! :D

I do say I compliment you aldo on your tidy and well kept code. Some people I've seen, *shudders*, have stuff all over the place with little to no comments.

I'll probably take a look at php, it seems fairly simple to use... is it? :)

HLP should have a code Academey... :p
Title: Parsing a text file
Post by: kasperl on March 29, 2005, 09:26:52 am
Can you do me a favour, and use the EINDE: DNS (minus the space, smiley prevention) as a start echo?

The existing program is already sent away, stupid as I am, and there are some existing logs already...
Title: Parsing a text file
Post by: aldo_14 on March 29, 2005, 09:30:49 am
I'll work something out, then.
Title: Parsing a text file
Post by: Sandwich on March 29, 2005, 03:27:16 pm
Quote
Originally posted by Grug
I'll probably take a look at php, it seems fairly simple to use... is it? :)


Yes, if you want to learn it up to a basic-medium level. Beyond that and you start delving into areas that are both quite powerful and quite confusing for someone with my brain structure. :p
Title: Parsing a text file
Post by: Grug on March 29, 2005, 04:08:18 pm
Quote
Originally posted by Sandwich


Yes, if you want to learn it up to a basic-medium level. Beyond that and you start delving into areas that are both quite powerful and quite confusing for someone with my brain structure. :p


Hehe. That an offer to help when I get stuck? :D :p
Title: Parsing a text file
Post by: Sandwich on March 30, 2005, 04:58:01 pm
No, that's a request for help when you surpass me. :p
Title: Parsing a text file
Post by: Grug on March 30, 2005, 04:59:17 pm
lol. Doubt that'll happen any time soon. :p

I'll trade my (amatuer) Java knowledge for (expert) php knowledge. ;)
Title: Parsing a text file
Post by: aldo_14 on March 30, 2005, 05:47:55 pm
Quote
Originally posted by aldo_14
I'll work something out, then.


Um... I'm really busy with work ATM - this is the last week of my job, I need to document stuff for whoever follows me developing the...thing.. - not sure when I'll be able to fix it.
Title: Parsing a text file
Post by: Grug on March 30, 2005, 05:53:19 pm
That would also explain your absence in the internals? ;)
Title: Parsing a text file
Post by: ZylonBane on March 30, 2005, 07:46:53 pm
Ahhh, nothing quite like a screenful of Java code to test the ol' gag reflex. What a horrid, bloatful language. "System.out.println" indeed. Blech. Ack. Blargh.
Title: Parsing a text file
Post by: Grey Wolf on March 30, 2005, 08:06:03 pm
The sad thing is it actually looks quite a bit neater than the C++ code I just wrote the other day in my Computer Science class.  330 lines or so just for Tic Tac Toe and some basic AI.  Far too many nested tables...
Title: Parsing a text file
Post by: aldo_14 on March 31, 2005, 02:20:21 am
Quote
Originally posted by Grug
That would also explain your absence in the internals? ;)


Pretty much, yes.

Quote
Originally posted by ZylonBane
Ahhh, nothing quite like a screenful of Java code to test the ol' gag reflex. What a horrid, bloatful language. "System.out.println" indeed. Blech. Ack. Blargh.


Thank you for your oh-so-valuable and helpful contribution.
Title: Parsing a text file
Post by: WMCoolmon on March 31, 2005, 02:44:46 am
Quote
Originally posted by aldo_14
No, it's a quick and dirty hack with exceptions........there are far better ways to do this sort of thing with recursion or tree-building or regular expressions, or really anything beyond a strict-format linear run.  But they take longer :D


You should take a look at the code for my ships.tbl reader I made awhile back. :drevil: Something I would call 'hackish' would not enforce format in any way, would just read the lines in and parse them. And it wouldn't have any globally defined constants; they'd all be tacked into the code.

I don't even know how to use exceptions in C++ :p I just (try to) keep them from happening (eg: Do not use a null pointer).

Although the whole exceptions thing strikes me as not having a place in a well-coded app, because generally you do want to keep that array from overflowing, or keep that null pointer from being used. Exceptions are (IMHO) like failing to avoid a fallen tree in the middle of the road because you have insurance. :p
Title: Parsing a text file
Post by: aldo_14 on March 31, 2005, 03:12:03 am
Quote
Originally posted by WMCoolmon


You should take a look at the code for my ships.tbl reader I made awhile back. :drevil: Something I would call 'hackish' would not enforce format in any way, would just read the lines in and parse them. And it wouldn't have any globally defined constants; they'd all be tacked into the code.

I don't even know how to use exceptions in C++ :p I just (try to) keep them from happening (eg: Do not use a null pointer).

Although the whole exceptions thing strikes me as not having a place in a well-coded app, because generally you do want to keep that array from overflowing, or keep that null pointer from being used. Exceptions are (IMHO) like failing to avoid a fallen tree in the middle of the road because you have insurance. :p


I use exceptions as a sort of cascading fallback-on-error control; easiest way AFAIK to simply halt a program on error (although there's not really a recovery strategy that can be used here; just carry on and print out what result there is up to the failpoint)...

basically, if there's an error which can't be handled at the local 'level' (i.e. in the called routine), I can use an exception to inform the callee and let it either perform recovery/reaction, or simply pass that exception on the next callee layer, etc, until something that handles the exception receives it;  what you're talking about is error avoidance, but exceptions are really for error detection and recovery anyways, which is a bit of an alternate paradigm - i.e. prevent the errors happening, but where there is the possibility of error, handle and recover (or fail gracefully) from it.

For example.... I'm working  (well, was working) on a distributed application that, in simple terms, requires nodes to communicate with each other via specially formatted string messages.  The nature of it is that each node sends messages to each other, and that each node is responsible for formatting the message prior to sending.  

On receipt, the receiver uses a method to extract the information it needs; there's the possibility due to code error on the other node (perhaps a version difference, or a 3rd party coder - it's a general infrastructure being developed rather than a single app, or simply corruption) that the string message format is dodgy.

If so, I can simply throw an appropriate exception from the method that obtains info from the string, that will inform the receiver node (callee) of the formatting error; I can't handle the error itself within the method, because the callee needs to know there is a format problem which is by nature unrecoverable.  I also need to skip everything affected by it, hence the use of a try/catch block...  point being, in this case I can't guarantee there is no error, because the data - the error source - comes from a 3rd party source and cannot be natively guaranteed as correct.

EDIT; not that I actually do error correction :D (actually, it's a concept prototype; the deliberate intent is to fail-on-error at present.  Not my intent, though......)

That's an example for proper use of exceptions, IMO :)
Title: Parsing a text file
Post by: WMCoolmon on March 31, 2005, 03:32:07 am
Wouldn't roughly the same thing be possible by returning the function with a special value to indicate failure/the error?
Title: Parsing a text file
Post by: aldo_14 on March 31, 2005, 03:46:19 am
Yeah, but then you'd be adding in specific handlers for each possible error return case as well as the allowed return; you'd also need to add in constants to check for for different error causes (i.e. instead of using an AttributeChangeException, InvalidChangeException or similar).

You'd also have to add in explicit backtracking to find the error cause and fetch the variable/s that cause it; using an exception (specifically designed subclass), you can encode the problematic data or any other info encapsulated within the error case.

Not to mention that you'd need to define a returned special value which cannot be generated by legal data - which in turn implies you need to make assumptions on what paramater values and return you can get.  And there's a side issue of typed returns; how do you define the error Object for a method returning type ArrayObject (or something)?  

(there's also possibly a memory issue if returning inherently large objects for the return, or perhaps if you are relying on multiple returns to determine an action... but I'm not sure of the comparative memory usage of an exception, however)

So... at a certain, simple level IMO you could use a special type return but you'd not have any real advantages I can think of over an exception; and at a higher level you'd lose the advantages of encapsulating the error type, data and cause in an exception object.

Bearing in mind this is from a Java and not C/C++ perspective of exceptions
Title: Parsing a text file
Post by: WMCoolmon on March 31, 2005, 04:07:02 am
Ahh, I see. I suppose in C you could use an error struct and a goto to do something like that, and in C++, right off the top of my head, you could create an error object class with an identifier int, then derive types from it, allocate the type, return a pointer to it and cast the pointer type in the caller function. Messy, but it'd be about as fast as you could get, aside from having things pre-allocated.

Whether or not that'd be faster than exceptions or not I don't know. It'd basically be the same thing, except not using any (hidden) global variables to pass the errors along.

Actually, I guess you could make an 'error' struct, then have a union (basically a multi-type object, with enough space to hold the largest object) with pointers to all the various error structs. You wouldn't have to cast pointers then, and it'd be valid in C/C++, and could still have the flexibility of the class method.

All of that, though, I doubt you could do in Java. To quote the one and only Java book I've read - "Java is a strongly-typed language".
Title: Parsing a text file
Post by: karajorma on March 31, 2005, 04:16:22 am
The other benifit of Java's Error handling system is that it forces you to think of possible errors that could easily be overlooked in C or C++

For instance any file access operation in Java forces you to handle the FileNotFoundException. How many C++ programs crash out simply because the programmer forgot to think of the possibility that the file might not be there when it was needed?

If you're writing a program and see an exception that can't picture a cause for it's often a big neon sign warning you that you've forgotten something.
Title: Parsing a text file
Post by: aldo_14 on March 31, 2005, 04:47:54 am
Quote
Originally posted by WMCoolmon
Ahh, I see. I suppose in C you could use an error struct and a goto to do something like that, and in C++, right off the top of my head, you could create an error object class with an identifier int, then derive types from it, allocate the type, return a pointer to it and cast the pointer type in the caller function. Messy, but it'd be about as fast as you could get, aside from having things pre-allocated.

Whether or not that'd be faster than exceptions or not I don't know. It'd basically be the same thing, except not using any (hidden) global variables to pass the errors along.

Actually, I guess you could make an 'error' struct, then have a union (basically a multi-type object, with enough space to hold the largest object) with pointers to all the various error structs. You wouldn't have to cast pointers then, and it'd be valid in C/C++, and could still have the flexibility of the class method.

All of that, though, I doubt you could do in Java. To quote the one and only Java book I've read - "Java is a strongly-typed language".


Offhand, that sounds (semantically) pretty similar to the existing Java setup... if I understand you right, you're suggesting some form of generic structure which contains links to structures for specific error types? (struct is methodless, isn't it?... been ages since I did C); i.e. relating to the specification/identification of the error type?

Java equivalent is loosely the inheritance hierarchy of the exception class; i.e. there's the generic Exception class, and subclasses (whose identity conveys a specific type or group of error/s) of that such as ArrayOOB, ClassCast, etc.  The try/catch black handles that ala
Code: [Select]

try {
//blah
}
catch(ArrayOOBException aooe)  {
//OOB specific stuff
}
catch(ClassCastException cce){
//cce stuff
}
catch(Exception e) {
//generic exception handling for any type not caught in above catch
}
finally {
//clause which occurs in exception or non-exception case (always)
//i.e. to cleanup on end*
}


From what I suggest, you're suggesting something very similar in effect, but not organisation.

As an aside, this (http://www.cluck.com/HTML_Articles/ExceptionHandling.html) seems interesting.  The stack trace for java exceptions is quite slow, for one thing, if exceptions are not used carefully.

*NB: I don't use finally much, myself, I usually forget :o... apparently in Java 1.1 there's a potential error if you have a try {} finally{} block, where exceotions can be lost...not sure if that still applies to newer java versions.  C++ - AFAIK - doesn't have an equivalent to finally because of the more explicit memory allocation/deallocation

EDIT; I should point out I only used the generic Exception supertype in the previous (parsing thing) code because I didn't feel the need to define a specific new type.... argueably could have defined something like a FormatException class.
Title: Parsing a text file
Post by: kasperl on March 31, 2005, 12:42:48 pm
OK, I think this thread is teaching me a lot more then I ever dreamed off, thanks, all of you.

The program at hand as been used, and used reasonably well, and the problem has been fixed. I think, a bit. I'm going to write a batch file to clean out the winXP Hosts file, and all should be reasonably well, as long as she uses Mozilla.


I'll be using the code in this thread to write any parser I think I  might be needing, if I need to do something quick. I think I can handle quick copy/paste/rip hacking.