Keep It Simple, Stupid PerlMonks

### A CDMA problem

by Fighter2 (Novice)
 on Nov 15, 2011 at 21:30 UTC Need Help??
Fighter2 has asked for the wisdom of the Perl Monks concerning the following question:

Hello.

First let me being by telling you thank-you, as because of you I am learning Perl, and I am grateful. I was here last week asking a question. I am hacking some code written by someone else. Unfortunately I must do this and get results and I am against a deadline. This is not for hw. Very real.

I find somethings very difficult to do, and hence I am here again for the 2nd time. I have progressed a LOT thanks to you guys in one short week. I took all your advice to heart and studied real hard.

Now comes the problem.

Here, please take a look at this logfile

Let's say from the logfile the order of the logpackets are as follows

0x1069,0x108A,0x108A,0x1069,0x108A,0x108A,0x1069,0x108A,0x108A,0x1069,...

Now, my problem is, how may I fetch the rxAgc0 and rxAgc1 from each log record of 1069 for every occurance of 108A. I need closest (or latest) two scalar variables from 1069 for every occurance of 108A to do some math (with some values inside 108A). Can you help me fetch the latest two values of rxAgc0 and rxAgc1 from 1069? How do I code that? The view array (in code below) is sorted in order of occurance of every log according to the timestamp. This is what I did so far (I need some of these values and references for other reasons, hence this way.) I have defined some of the important functions below for you to get an idea of the hash structure.p.s: I don't know why the code wraps around.

```my \$nff = new NffInterface;  #Initializes some variables and starts a
+software.
my \$logid = \$nff->loadLog(\$inputFile);   #\$inputfile is the logfile.
my \$filter1={type => "logcode", logcode => 0x1069};
my \$fid1=\$nff->createFilter(\$logid,\$filter1);  #Picks an index in the
+log
my \$filter2={type => "logcode", logcode => 0x108A};
my \$fid2=\$nff->createFilter(\$logid,\$filter2);
my \$view=\$nff-> createView([\$fid1,\$fid2]); #Creates an array of multip
+le filters on order of the particular logId.

my (\$obj,\$obj1,\$obj2,\$tmp,\$myc2i,\$codetype);
\$myc2i=0;

do{
\$tmp=\$nff->getNextLogFromView(\$view); #Iterates on filter indices
if(\$tmp ne -1){
if(getLogCode(\$tmp) eq 0x108A){
\$codetype="0x108A";
#print \$codetype;
\$obj1 = parse108A(\$tmp);   #Defined below. Parses 108A log
+s.
my @pns = keys(%{\$obj1->{sectors}});

for(my \$i=0;\$i<\$obj1->{numSectors};\$i++){
#for each sector
my \$sector = \$pns[\$i];
my \$currentSectorPower=0;
for(my \$j=0;\$j<\$obj1->{sectors}->{\$sector}->{numFingers};\$
+j++){
\$currentSectorPower=\$currentSectorPower+ \$obj1->{se
+ctors}->{\$sector}->{\$j}->{c2i};
}
\$obj->{sectors}->{\$sector}->{cumulativePower}=\$currentSect
+orPower;
\$myc2i=\$obj->{sectors}->{\$sector}->{cumulativePower};
#print \$myc2i;

if(getLogCode(\$tmp) eq 0x1069){
\$codetype="0x1069";
#print \$codetype;
\$obj2 = parse1069(\$tmp);   #Defined below. Parses out 1069
+ logs.
for (my \$j=0;\$j<2;\$j++){
for my \$rxAgc0(%{\$obj->{\$j}}) {
print "\$rxAgc0 = \$obj2->{\$j}{\$rxAgc0}\n";
#print Dumper \$obj2;
}
print "\n";
}
}
}
} while(\$tmp ne -1);

There are more details below. \$c2i is the value we need from every finger of 108A for each PN. I need to add something to this value. But I cannot do so unless I have fresh rxAgc0 and rxAgc1 from every closest 1069 logpacket for every 108A. :(

Thank you very much for wanting to help me. I am very much in debt to you. Also so many thanks to davidofor teaching me a lot, ikegami telling me about this place, and aaron_baugherbe patient and easy on me. I really need your brains here everybody. :(

See here if you want in detail the info on the logfile

I must explain this part in detail so as not to confuse those who wishes to help me. We can both help each other. I will teach you the CDMA Fading Engine. You can help me with understanding, how to code. Here, look at this picture.

There are only two different logs there. And thousands of records of each. The two types of log are 0x108A and 0x1069. 108A occurs every 30ms and 1069 occurs every 53ms. The log records were collected for closer to an hour by a system(and software) attached to a cellphone while travelling. the logcode 0x108A corresponds to the logdata that corresponds to multipath SNR (RSSI) received by the cellphone. We call it c2I value and they are in dB. Its a ratio and not power. Usually a cellphone receives upto 0-8 different multipaths from a Single sector or basestation, which we call PN's. And multipaths are called Fingers. Here is an example of one sector (or PN) with few fingers (or multipaths) of log type 0x108A. This is logrecord #5, collected at 2011 Feb 5 at 01:24:19.148. Here's the link. In that example there are 4 Fingers, or multipaths from ONE single sector which is PN 52 (remember PN is sectorID). However, in real world a cellphone can receive multipaths from other sectors as well. Here is an example of such a scenario. In this example, you can see fingers (or multipath RSSI's) from 2 different sectors, of sector ID's 52 and 56. Upto 6 sectors is possible. So in a nutshell, 0-6 sectors is possible to be tracked by a cellphone and 0-8 fingers per sector can be received.

Now pay attention to the bottom of those. You will see the actual contents inside the logfile above. The logfile is in .dlf format, and even if I give it to you, you cannot open it and view its contents unless you have a special software which dencrypts. Hence why I had to take a screenshot of that logfile to show you. But essentially, it contains a bunch of HEX values which are in Little Endian format for each logtype. Log type of 0x108A is 134bytes log, which is 2bytes of length info, 10 bytes of header and the rest is all payload. You can see that from the picture I have linked above. It is one logpacket opened using that special software I mentioned, which parsed out and converted the HEX values and displayed that info. But we don't have that privilege when it is just the logfile.

Now the other type of log packet that log collected is of the type 0x1069. This occurs every 53ms. This packet contains info about actual Rx power received by the cellphone in dBm. Here is what in that data. This is from log record #6. It is again opened by the software which parsed and converted to view. So both types logs are collected simultaneously onto one logfile. THe only index for each log irrespective of logtype is the timestamp. And we have a file full of them in HEX format. Which I need to parse out.

And see below for the structure of the hash from the subroutine to parse 0x1069 type of packets

```#0x1069
sub parse1069{
my \$data=shift;
my \$len=length(\$data);
my (\$obj,\$tmp,\$j,\$k);
\$obj->{timestamp}=getTimestamp(\$data);
\$len=\$len-12;
(\$tmp,\$data)=unpack("a12 a\$len",\$data);
for(\$j=0;\$j<2;\$j++){
\$len=\$len-13;
(\$tmp, \$obj->{\$j}->{txOpenLoop},\$obj->{\$j}->{txClosedLoop},\$ob
+j->{\$j}->{txPilot}, \$obj->{\$j}->{txTotal}, \$obj->{\$j}->{rxAgc0}, \$obj
+->{\$j}->{rxAgc1}, \$data)=unpack("C v v v v v v a\$len",\$data);
\$obj->{\$j}->{paState} = (int(\$tmp/32)) % 2;
\$obj->{\$j}->{rachetMode} = (int(\$tmp/16)) % 2;
if(\$obj->{\$j}->{txOpenLoop} >= 2**15){ \$obj->{\$j}->{txOpenLoop
+} -= 2**16;} \$obj->{\$j}->{txOpenLoop} /= 256;
if(\$obj->{\$j}->{txClosedLoop} >= 2**15){ \$obj->{\$j}->{txClosed
+Loop} -= 2**16;} \$obj->{\$j}->{txClosedLoop} /= 256;
if(\$obj->{\$j}->{txPilot} >= 2**15){ \$obj->{\$j}->{txPilot} -= 2
+**16;} \$obj->{\$j}->{txPilot} /= 256;
if(\$obj->{\$j}->{txTotal} >= 2**15){ \$obj->{\$j}->{txTotal} -= 2
+**16;} \$obj->{\$j}->{txTotal} /= 256;
if(\$obj->{\$j}->{rxAgc0} >= 2**15){ \$obj->{\$j}->{rxAgc0} -= 2**
+16;} \$obj->{\$j}->{rxAgc0} /= 256;
if(\$obj->{\$j}->{rxAgc1} >= 2**15){ \$obj->{\$j}->{rxAgc1} -= 2**
+16;} \$obj->{\$j}->{rxAgc1} /= 256;
}
return \$obj;
}

Please don't worry. if(\$obj->{\$j}->{rxAgc0} >= 2**15){ \$obj->{\$j}->{rxAgc0} -= 2**16;} \$obj->{\$j}->{rxAgc0} /= 256; is converting an unsigned 16 bit number (acquired via unpack with v as the format code, which is 16 bit little-endian unsigned) to signed and then dividing by 256. The reason is the decimal value of that HEX, say val/256 gives us the RSSI in dB per finger. The author could have just used a signed format to unpack and skipped this, but I am not going to rewrite all this mess and hence its just fine to leave it as such. The important point is this. You got the structure of the hash, and the fact that this is stored in a subroutine in a sub.pm file which is use'd in my logparser.pl file. Ok. I hope you're still with me.

And see below for the hash structure of 108A packet

```sub parse108A {
my \$data=shift;
my \$len=length(\$data);
my (\$obj,\$tmp,\$j,\$k);
\$obj->{timestamp}=getTimestamp(\$data);
\$len=\$len-12;
(\$tmp,\$data)=unpack("a12 a\$len",\$data);
my \$fingId;
\$len = \$len-10;
(\$obj->{srchState},\$obj->{MSTR},\$obj->{MSTRError},\$obj->{MSTRPilot
+PN},\$obj->{numF},\$data)=unpack("C V v v C a\$len",\$data);
my (\$pilotPN,\$rtcOffset,\$c2i,\$res1,\$res2,\$ant0c2i,\$ant1c2i);
\$obj->{numSectors}=0;
for(\$j=0;\$j<\$obj->{numF};\$j++){
\$len=\$len-14;
(\$pilotPN,\$rtcOffset,\$c2i,\$res1,\$res2,\$ant0c2i,\$ant1c2i,\$data)
+ = unpack("v V v C C v v a\$len",\$data);
if(\$pilotPN < 1024){
if(not defined(\$obj->{sectors}->{\$pilotPN})){
\$obj->{sectors}->{\$pilotPN}->{numFingers}=0;
\$obj->{numSectors}++;
}
\$fingId=\$obj->{sectors}->{\$pilotPN}->{numFingers};
\$obj->{sectors}->{\$pilotPN}->{\$fingId}->{c2i} = \$c2i;
if(\$rtcOffset > 2**31-1){
\$obj->{sectors}->{\$pilotPN}->{\$fingId}->{delay} = \$rtc
+Offset-2**32;
}
else{
\$obj->{sectors}->{\$pilotPN}->{\$fingId}->{delay} = \$rtc
+Offset;
}
\$obj->{sectors}->{\$pilotPN}->{numFingers}++;
}
}
return \$obj;
}

This is the finger packet. \$c2i is the value we need from every finger of 108A for each PN. I need to add something to this value. But I cannot do so unless I have fresh rxAgc0 and rxAgc1 from every closest 1069 logpacket for every 108A. :(

Replies are listed 'Best First'.
Re: A CDMA problem
by juster (Friar) on Nov 16, 2011 at 22:01 UTC

Well, I will take a stab at this. I fed my coding addiction by messing with your example. I ran it through perltidy because I had a hard time reading it. After reading it and seeing how simple it was I rewrote some of it for my own edification.

Your post is way too in depth. I don't really have time to care about CDMA. I generally simplify problems by looking at the data coming in and going out, getting munged, or whatever. You have a stream of packets. Each packet has a timestamp and data members, not all of which you will use. You only want to compare data from packets with timestamps close to each other, right? (This is an important fact you weren't specific about). Namely you want to compare c2i data from 108A/finger/multilink packets with rxAgc0-rxAgc1 data from 1069 packets. Yum.

You can either do some awkward logic where each time you see a 1069 packet you backtrack to compare it with every old 108A packet and vice versa if you see a 1069 packet. You would also have to remember to discard old packets.

Or you could just create an array of data which you map from your original stream of raw packets. To conserve memory this array contains only the data you need. The advantage over the stream is that you can go backwards and forwards so the logic is also much easier.

Stream in. Data out. Poke around the data, do some calculations. Discard data. Spit out the calculations.

Of course, I can't test this to see if it works. Hopefully you get the idea and adapt it to your needs. Or you've probably moved on after a day and won't see this.

Regarding the parsing of the raw packets, you can specify little endianness by using the '<' (less-than) character. You can specify a repeat count with a number. Yay fun example: perl -e 'print pack(q{s<2}, 0x0A0D, 0x0A0D)'; (prints "\x0D\x0A\x0D\x0A" aka "\n\n" in windows). Parenthesis create sub-groups. So your parse1069 unpack template could be: a12 (C s<6)2. I think so anyways, pack is tricky... Yes you said you didn't want to change it but it's not much work and there's nothing more productive than deleting lines of code ;-). Consider using a module like Convert::Binary::C.

You also might want to consider using arrayrefs instead of hashrefs which have weird entries that help them pretend to be arrays. Consider:

{ 'count' => 2, '0' => 'wtf', '1' => '?' } vs [ 'wtf', '?' ]

Juster. Thank you so very much for trying out my problem. I am indeed grateful to you. I will go home and run this, and tell you what results I get. Boy, if you do like problems that challenge and stretch your mind, we have plenty, which is a continuation of this one. What you attempted was a small part of a big project. Anyways, I will keep you in loop. Thank-you much for being kind and helpful. Grateful to you. Thanks bud (:

Hi juster.

So my code compiles. and thanks for cleaning it up so nicely. I do have an error when I try to run it though.

```C:\Dropbox\PandaDBCM>perl -d logprocessTempNff.pl sample2.nff out.txt
1069

Can't use string ("980904258.881354") as a HASH ref while "strict refs
+" in use at logprocessTempNff.

pl line 222.

I think "980904258.881354" is the timestamp. It looks like it. I know it will take a few tries to make this work. but how do I fix this error I wonder. Hmm. ?

The error is pretty straightforward. That happens when you try to use a string as a hash reference. I.e.:

```my \$foo = 'bar';
\$foo->{'key'} = 'value';
# or likely
for my \$k (values %{\$foo}){ ... }

Just look at the line number specified and back-track from there. Use Data::Dumper or Data::Dump to ensure you have the values you are expecting (like a hashref instead of a string). Print them to screen alot, etc.

I know there is one typo where I used the wrong variable (\$logpkt should be \$fngpkt in one spot) and I think I might have used values() where I shouldn't have. Make sure you understand values. Make sure you understand references, read perlreftut for example.

Create A New User
Node Status?
node history
Node Type: perlquestion [id://938256]
Approved by Corion
help
Chatterbox?
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others perusing the Monastery: (7)
As of 2018-04-24 14:49 GMT
Sections?
Information?
Find Nodes?
Leftovers?
Voting Booth?
My travels bear the most uncanny semblance to ...

Results (86 votes). Check out past polls.

Notices?