Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change

Re^4: how to output file in Unicode

by anakin30 (Acolyte)
on Aug 24, 2012 at 19:47 UTC ( #989629=note: print w/replies, xml ) Need Help??

in reply to Re^3: how to output file in Unicode
in thread how to output file in Unicode

Hi remiah

I run a Perl script to extract user attribute values from LDAP.

Extracted output file contains First Name and Last Name of German users.

Supposed Last Name should be displayed as "Bjrn", but it is displayed as "Björn", which seems to be unrecognizable.

I learned that the output file was created in ASCII format should be created in Unicode UTF-8 format in order the German character to be visible.


Replies are listed 'Best First'.
Re^5: how to output file in Unicode
by remiah (Hermit) on Aug 25, 2012 at 04:13 UTC

    Here I suppose the LDAP module that returns last name, gives you "decoded utf8 string". What do you see in log.txt file? and any warning message?

    my $lastname=''; #here put the return value of LDAP open( my $fh, ">:encoding(UTF-8)", "log.txt") or die $!; print $fh utf8::is_utf8($lastname) ? 'utf8 flagged,' : 'not utf8 flagg +ed,'; print $fh $lastname; close $fh;

    Some string (in this case LDAP)
    LDAP module?
     |        <--- here LDAP returns what? is important information
    utf8 text
    Do you see the manual of LDAP module has 'encoding' or 'binmode' section for this information?

      I have written this script below,

      the output file generated but i still seeing, German "Umlaut" in the csv output file. Example : Björn

      why the csv output file is not encoded into encoding(UTF-8)??, am i missed any steps here!!!!

      #!/usr/bin/perl use Net::LDAP; use Encode; use Net::LDAP::Entry; use Net::LDAP::Control::Paged; use Net::LDAP::Constant qw( LDAP_CONTROL_PAGED ); #$outputfile = "LicMan1.csv"; $outputfile = 'C:\Meta\MDS\data\out\LicMan1.csv'; #$inputfile = 'D:\Meta\MDS\data\out\LicMan1.csv'; $outputfile2 = 'C:\Meta\MDS\data\out\LicMan.csv'; open (FH, ">$outputfile") or die "$!"; print FH "import_id;import_data_source_id;login;import_level_2_id;impo +rt_location_id;last_name;first_name;email;is_active;language;remarks; +\n"; my @args = ( base => $base, scope => 'sub', filter => $query, attrs => \@attrs, control => [ $page ],); my $cookie; while (1) { $mesg = $ldap->search ( @args ) or die $!; while (my $entry = $mesg->shift_entry()) { #uid,dpwncrestcode,dpwnc,sn,givenname,mail,dhlaccountstatus,preferredl +anguage,dhlemployeetype my $entrydn = $entry->dn(); my $uid = $entry->get_value('uid'); my $dpwncrestcode = $entry->get_value('dpwncrestcode'); my $dpwnc = $entry->get_value('dpwnc'); my $sn = $entry->get_value('sn'); if (($sn eq "") || ($sn eq " ")){ $sn = "-"; } else { $sn = "$sn";} my $givenname = $entry->get_value('givenname'); if (($givenname eq "") || ($givenname eq " ")){ $givenname = "-"; } else { $givenname = "$givenname";} my $mail = $entry->get_value('mail'); my $dhlaccountstatus = $entry->get_value('dhlaccountstatus'); if ($dhlaccountstatus == "enabled") { $dhlaccountstatus = "1"; } else { $dhlaccountstatus = "0";} my $preferredlanguage = $entry->get_value('preferredlanguage'); if ($preferredlanguage ne "de") { $preferredlanguage = "en"; } else { $preferredlanguage = "de";} my $dhlemployeetype = $entry->get_value('dhlemployeetype'); if ($dhlemployeetype eq "") { $dhlemployeetype = ";"; } else { $dhlemployeetype = "$dhlemployeetype";} print FH "$uid;dpdhl_eds;$uid;$dpwncrestcode;$dpwnc;$sn;$givenname;$ma +il;$dhlaccountstatus;$preferredlanguage;$dhlemployeetype\n"; } } $ldap->unbind; close(FH); open (INPUT, "<:encoding(UTF-8)", $outputfile) or die "Cannot open fil +ename for input: $!"; while (<INPUT>) { s/"//g; $replace .= $_;} close INPUT or die "Cannot close filename: $!"; open (OUTPUT, ">:encoding(UTF-8)",$outputfile2) or die "Cannot open fi +lename for output: $!"; print OUTPUT $replace; close OUTPUT or die "Cannot close filename: $!";
        So, what's going on if first open, let it open with utf-8 io layer?
        open (FH, ">$outputfile") or die "$!";
        revise to
        open (OUTPUT, ">:encoding(UTF-8)",$outputfile or die "$!";
        What is the result?

        Second question: Is that $givenname?

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://989629]
[1nickt]: ambrus perhaps you know where the keys are?
[ww]: OTOH, -- for ads suggesting that being good at games is somehow eq "greatness"
[ambrus]: 1nickg: obviously the gamers don't know. but anyway, I need the whole room backlit, because watching a monitor in a dark room (like many people do) is tiring for my eyes, even more so than doing it in a lit room.
[1nickt]: ambrus I agree. I suspect one's eyes are always trying to adjust across the boundary between very bright and very dark ... like a camera's autofocus struggling to choose the target.

How do I use this? | Other CB clients
Other Users?
Others surveying the Monastery: (18)
As of 2017-03-27 13:55 GMT
Find Nodes?
    Voting Booth?
    Should Pluto Get Its Planethood Back?

    Results (320 votes). Check out past polls.