Re^3: Fetching table from website using HTML::TableExtract

[shmem]

Corion doesn't mean you fix the data in the webpage directly, but after you have extracted it with HTML::TableExtract which is properly inserting empty fields for data cells skipped due to colspan or rowspan. But you have multiline fields:

The second row of your $ts->rows is

Region,Level 31.03.2016,,Sanction/Renewal
01.04.2016 to 28.02.2017,,,Level 28.02.2017,,Sanction/Renewal
During Current Month

         ,,,Level 26.03.2017,,Growth
as on
26.03.2017,

After Level 31.02.2016 there's an empty field because of colspan="2". The next field is

Sanction/Renewal
During Current Month

so all you have to do is removing trailing whitespace/newlines from each field:

foreach my $ts ( $te->tables ) {
    print "Table (", join( ',', $ts->coords ), "):\n";
    foreach my $row ( $ts->rows ) {
       s/[\s\n]+\z/ for @$row; # <--- here
       # s/\n/ /gs for @$row; # uncomment if you want to convert
                              # multiline fields into single line
       $OUT-> print( join( ',', @$row ), "\n");
    }
}

[sachin raj aryan]

Hi i tried your code somewhat working but still one issue below is the output

Amount in Crores,,,,,,,,,,,,,,

Region,Level 31.03.2016,,Sanction/Renewal01.04.2016 to 28.02.2017,,,Level 28.02.2017,,Sanction/RenewalDuring Current Month,,,Level 27.03.2017,,Growthas on27.03.2017,

if you see multiple commas in first line and more than one in second row

[shmem]

if you see multiple commas in first line and more than one in second row

What do you expect?
You are basically outputting a CSV file in which each line has the same amount of fields, separated by commas. Nothing between two commas is an empty field produced by expanding field spanning captions of the original table denoted as e.g. colspan="2" in html.

If those fields were omitted, the placement of the captions would not correspond to the columns of the table data. Load the output file as a spreadsheet (e.g. in Excel) and you will see the effect.

I don't know the purpose of what you are doing. If you want to remove empty fields from the output, proceed as follows:

foreach my $ts ( $te->tables ) {
    print "Table (", join( ',', $ts->coords ), "):\n";
    foreach my $row ( $ts->rows ) {
       s/[\s\n]+\z/ for @$row;
       s/\n/ /gs for @$row;
       $OUT-> print( join( ',', grep { length $_ } @$row ), "\n");
    }
}

Again, load the data into a spreadsheet to see the difference. Compare to the original table on the website.

[sachin raj aryan]

thanks it worked with first solution , when i opened it in excel ..