Weird error with Cryprt::Blowfish and Crypt::DES

[Beechbone]

We're using Net::SSH::Perl here and some time the last weeks something changed and we always get an error in one of our modules:

input must be 8 bytes long at /usr/local/perl/lib/site_perl/5.8.4/i686
+-linux/Crypt/DES.pm line 57.

Same happens when forcing Blowfish. I tracked the program execution down to this (DES code is identical):

Crypt/Blowfish.pm:
        Crypt::Blowfish::crypt($data, $data, $self->{'ks'}, 0);

Crypt/Blowfish.xs:
void
blowfish_crypt(input, output, ks, dir)
        char *  input = NO_INIT
        SV *    output
        char *  ks = NO_INIT
        STRLEN  input_len = NO_INIT
        STRLEN  output_len = NO_INIT
        STRLEN  ks_len = NO_INIT
        int             dir
        CODE:
        {
                input = (char *) SvPV(ST(0), input_len);
                if (input_len != 8)
                        croak("input must be 8 bytes long");

We added debug code around the Perl code and it clearly states that $data is exactly 8 bytes long when given to the C function. It is a real string as as sure as $data = "$data"; can make. The data is just normal 8 bytes, I see no pattern in it, and it is not always the same data. One example would be (in hex): "0a ac 30 c6 f8 b1 e7 b9".

I'm clueless, any ideas?

[gaal]

Try adding a debug print in c, closer to where the croak happens?

        CODE:
        {
                input = (char *) SvPV(ST(0), input_len);
                printf("handling %d bytes\n", input_len);
                if (input_len != 8)
                        croak("input must be 8 bytes long");

[Beechbone]

Ok, got a result. input_len is 12. At least it was 12 in the single test run we managed to complete before lunch. Next thing I'll try is:

        if (input_len != 8) {
            printf("blowfish_crypt(): handling %d bytes\n", input_len)
+;
            printf("Bytes: ");
            for (i = 0; i < input_len; i++) {
                unsigned char a = (unsigned char) *(input + i);
                printf("%d ", a);
            }
            printf("\n");
            croak("input must be 8 bytes long");
        }

But again I have to wait. Seems I'm the only person around who's not at lunch right now...

PS: I know that my C code is ugly, but at least it works. ;-)

[Beechbone]

Not yet, (thanks for the syntax, my C is very weak ;)), we need someone with access to the compiler machine first...

Hopefully tomorrow we can try it.

---

Update:

We already know that input_len is not 8 bytes, so the exact value is not that important. Does anyone have C code to dump the complete SV? That would be much more useful, I'd say...

[dmorgo]

How do you know that $data is exactly 8 bytes long? In other words, exactly what perl code are you using to determine that? Please share that snippet.

It may be the data is exactly 8 characters long, but not 8 bytes. bytes != characters. Maybe the data is encoded in utf8?

If you're reading the data from a file handle, try doing binmode(handle) before reading in the data.

[Beechbone]

Got it:

Perl:   a7 53 64    81   c6  0a 30    b9
C:   c2 a7 53 64 c2 81 c3 86 0a 30 c2 b9

So the internal strings of Net::SSH::Perl got "infected" with the UTF-8 bit. Seems some of the strings it gets are declared in a "use utf8" area (but are 7bit themselves).

Now, as what Net::SSH::Perl gives to Cypher::* are binary data (byte strings) regardless of the input strings (character strings), how would I best fix that?

I see three possibilities:

1) Fix Net::SHH::Perl to make a difference between character strings it gets and the byte strings it uses to build the SSH protocol in it.

2) Change our code to disable the UTF8 bit on every string that is given to Net::SSH::Perl.

3) Change Crypt::* to turn the UTF8 bit off on data that is 8 characters long.

I'd say #1 would be the cleanest way and #3 the fastest. As we have a deadline of this afternoon for a complete program run, I will implement #3 as a workaround fast. But, what should we do on the long run?

BTW: Thank you to all who helped, tried to help, or would have helped if they could.

---

Update:

HELP, How DO I do this? I've got the String as above labeled 'Perl' with UTF8 on. When I just disable the flag, I get the 'bad' String, as labelled with 'C'. So how do I convert that string to a string of 8 bytes with the values as they should be?

---

Update2:

Ok, I found that, too. utf8::downgrade($data);

[Anonymous Monk]

Thank you soooo much. I worked hours on that problem without finding a reason or solution for that. I read my plaintext string from a xml file and "utf8::downgrade($data);" did the trick.

[druidmatrix]

Thank you very much for this solution. I had the same issue with "input must be 8 bytes long".

Strangely enough, I seem to only get this problem when I use strings coming from XML::Simple (XML::Parser), not when I try to run the same commands on CLI or from a different script that also uses Net::SSH::Perl, but gets the parameters from CLI args.

[Beechbone]

  my $d = $data;
  $d =~ s/(.)/sprintf("%x ", ord($1))/ge;
  print "data in hex: '$d'\n"

The data comes from Net::SSH::Perl, I'm not quite sure which part it is when the crypting fails, because there are so many levels of code above it, it's either the login or a command execution (SSH2).

When I look at aboce's output, I see that it's not the first execution that fails. I haven't counted it, but there have been at least 30 executions before the failing one.

[Anonymous Monk]

I've run into this problem too in the past. If you ever parse text from an XML document and try to send it via Net::SSH::Perl, you will likely run into the "8 bytes" problem. I think it's because XML by default is encoded as UTF-16 or something like that. I used this method: Encode::from($text,'utf16','iso-8859-1'); where $text is the data/command I send using Net::SSH::Perl.

[Anonymous Monk]

Many thanks I had the same problem when using XML::Simple. Encode::from_to($remotePass,'utf8','iso-8859-1'); did the "magic"

[Anonymous Monk]

Thanks it works with the tips above :)