Beefy Boxes and Bandwidth Generously Provided by pair Networks
Don't ask to ask, just ask
 
PerlMonks  

Re^3: Fetching table from website using HTML::TableExtract

by shmem (Chancellor)
on Mar 29, 2017 at 14:23 UTC ( #1186385=note: print w/replies, xml ) Need Help??


in reply to Re^2: Fetching table from website using HTML::TableExtract
in thread Fetching table from website using HTML::TableExtract

Corion doesn't mean you fix the data in the webpage directly, but after you have extracted it with HTML::TableExtract which is properly inserting empty fields for data cells skipped due to colspan or rowspan. But you have multiline fields:

The second row of your $ts->rows is

Region,Level 31.03.2016,,Sanction/Renewal 01.04.2016 to 28.02.2017,,,Level 28.02.2017,,Sanction/Renewal During Current Month ,,,Level 26.03.2017,,Growth as on 26.03.2017,

After Level 31.02.2016 there's an empty field because of colspan="2". The next field is

Sanction/Renewal During Current Month

so all you have to do is removing trailing whitespace/newlines from each field:

foreach my $ts ( $te->tables ) { print "Table (", join( ',', $ts->coords ), "):\n"; foreach my $row ( $ts->rows ) { s/[\s\n]+\z/ for @$row; # <--- here # s/\n/ /gs for @$row; # uncomment if you want to convert # multiline fields into single line $OUT-> print( join( ',', @$row ), "\n"); } }
perl -le'print map{pack c,($-++?1:13)+ord}split//,ESEL'

Replies are listed 'Best First'.
Re^4: Fetching table from website using HTML::TableExtract
by sachin raj aryan (Acolyte) on Mar 30, 2017 at 08:25 UTC
    Hi i tried your code somewhat working but still one issue below is the output

    Amount in Crores,,,,,,,,,,,,,,

    Region,Level 31.03.2016,,Sanction/Renewal01.04.2016 to 28.02.2017,,,Level 28.02.2017,,Sanction/RenewalDuring Current Month,,,Level 27.03.2017,,Growthas on27.03.2017,

    if you see multiple commas in first line and more than one in second row
      if you see multiple commas in first line and more than one in second row

      What do you expect?
      You are basically outputting a CSV file in which each line has the same amount of fields, separated by commas. Nothing between two commas is an empty field produced by expanding field spanning captions of the original table denoted as e.g. colspan="2" in html.

      If those fields were omitted, the placement of the captions would not correspond to the columns of the table data. Load the output file as a spreadsheet (e.g. in Excel) and you will see the effect.

      I don't know the purpose of what you are doing. If you want to remove empty fields from the output, proceed as follows:

      foreach my $ts ( $te->tables ) { print "Table (", join( ',', $ts->coords ), "):\n"; foreach my $row ( $ts->rows ) { s/[\s\n]+\z/ for @$row; s/\n/ /gs for @$row; $OUT-> print( join( ',', grep { length $_ } @$row ), "\n"); } }

      Again, load the data into a spreadsheet to see the difference. Compare to the original table on the website.

      perl -le'print map{pack c,($-++?1:13)+ord}split//,ESEL'
        thanks it worked with first solution , when i opened it in excel ..

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1186385]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others pondering the Monastery: (4)
As of 2022-05-19 18:54 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Do you prefer to work remotely?



    Results (72 votes). Check out past polls.

    Notices?