Random results from serin qualifier

[jmumby]

Hello I am using the following on a 28x1 with 16Mhz resonator. I am receiving a data string about 50 bytes long and trying to write to the scratch pad as a sort of serial buffer.

(some code to start device sending)

SERIN 1,T9600_16,B1,B2,B3,B4,B5,B6,B7,B8,B9,B10 'GET FIRST 10 BYTES

INC B20 'INCREMENT SCRATCH PAD POINTER 
PUT B20,B1 'WRITE BITE TO SCRATCH PAD
INC B20
PUT B20,B2
INC B20
PUT B20,B3
INC B20
PUT B20,B4
INC B20
PUT B20,B5
INC B20
PUT B20,B6
INC B20
PUT B20,B7
INC B20
PUT B20,B8
INC B20
PUT B20,B9
INC B20
PUT B20,B10

READIN:

(some code to start device sending)
'QUALIFY NEXT 10 BYTES WITH PREVIOUS 4 BYTES
SERIN [2000,RETRY],1,T9600_16,(B7,B8,B9,B10),B1,B2,B3,B4,B5,B6,B7,B8,B9,10

INC B20
PUT B20,B1
INC B20
PUT B20,B2
INC B20
PUT B20,B3
INC B20
PUT B20,B4
INC B20
PUT B20,B5
INC B20
PUT B20,B6
INC B20
PUT B20,B7
INC B20
PUT B20,B8
INC B20
PUT B20,B9
INC B20
PUT B20,B10
GOTO READIN

RETRY:
INC B19
IF B19 > 10 THEN END_OF_STRING 
GOTO READIN

This is pretty much what I use (actual code on another computer) and for the most part it works pretty good. Except it does on some instances fail 10 times even though it does not reach the end of the string. I have a counter on the actual program and it will sometimes fail 2 or 3 times with the qualifier yet work of the 4th or 5th attempt even though it is using the same data!

Im reading in mac addresses so the chances of the previous 4 bytes being the same are pretty remote. If I try and qualify by all 10 bytes it never works.

Why would this be? Im thinking the device is sending data before serin get's it's A into G so misses the first few bytes but this would fail every time wouldn't it?

Cheers

 

[BCJKiwi]

Suggest this form of code;
1. use the ptr form instead of B20
2. use the @ptrinc, e.g.

ptr = addr '(to set scratchpad location to start)
for b1 = 1 to 10
@ptrinc = b1 'WRITE BYTE TO SCRATCH PAD & INCREMENT SCRATCH PAD POINTER after write
next

@ptrinc an be used in this way on 28X1 fw A.2 but not fw A.1

From Manual 2 "Qualifiers are used to specify a ‘marker’ byte or sequence. The command serin 1,N2400,(“ABC&#8221,b1 requires to receive the string “ABC” before the next byte read is put into byte b1"
The qualifier is part of the data stream being read in but is not included in the data stored.

So unless the first 4 bytes are the same for all your MAC addresses it will fail to store any data as the qualifier will not be seen.

The first 3 bytes will be the same for every MAC address from the same manufacturer as the first part of the mac address is issued to the manufacturer and is therefore the same for all their devices. Part of the next 3 bytes are also likely to be the same if batches of NICs are from the same manufacturer as these are effectively a serial number.

Don't know what the test data is like but would expect the routine you have to work if and only if you have the same data repeated.

Also, since a mac address is 6 bytes long, perhaps only 6 bytes should be stored each cycle.

 

[moxhamj]

The serout code might be an issue. Say you send 50 bytes in a row. You read in the first 10 and then go off and store these bytes. But the data is still arriving. So you might have to cycle through all 50 bytes to get back to the beginning before the qualifier bytes match up again. It ends up being slower.

The maximum packet size is the 14 b0-b13 registers. There would be good arguments for using the last two bytes as some sort of checksum - either the sum of all the bytes or two unique characters or something. You might use the first byte as the address to store the data. So that leaves 11 bytes left - maybe make it a round 10 and use 2 bytes at the beginning.

Then add a qualifier to signify the beginning of a packet, eg ABC.

Then the syntax will be serin [baudrate],[pin],("ABC"),b0,b1...b12,b13
Check b12/13 are a valid checksum
Then store the data in b0/b1 location

And make sure there is enough of a delay between packets so the receiving picaxe can store each packet. Or use a hardware handshake via a spare pin.

 

[hippy]

I guess this is grabbing whatever comes back from some Wi-Fi adapter or other as you do your 'drive-by hunting for access points', so don't have a lot of control over what is received or when.

I think BCJKiwi nails it in that data is being dropped after you read the first few bytes, go away then come back for more. At that point it loses synch or sees data which matches the qualifiers but isn't the data packet you want. I also don't see anything which ensures that the first SERIN gets what you are expecting ( viz there are no qualifiers there ).

I'm not sure what to recommend. Handling high-speed unsolicted serial is a difficult task on a PICAXE which runs incredibly slowly compared to the data arrival rate. I think the first thing is to detail what the packets are which you are receiving and what it is you need to extract from them. If I were doing this on a PC I'd be trying to grab an entire packet and then parse that to extract the data from it.

 

[Technical]

You'll be much better off using hserin straight into the scratchpad. Look up hsersetup and hserin in the manual (part 2).

 

[jmumby]

Im way better off using Hserin, that is pretty cool.

The data I actually get from the unit is as follows

01-18-4D-5E-46-F8, -58dBm, I , NONE, The_quick_brown_fox_jumped_over_

<----------------------------70 Bytes---------------------------------->

Now a problem I always seem to have is -46- in the MAC address is always either -@6 or - 6- or -|6-. It is actually the 15th byte as by the time hserin starts it has missed the first two bytes which are garbage anyway.

So this occupies 70 bytes which leaves me plenty for latitude,longitude UTC and altitude. So my theory is to do a regular serin for the GPS stuff which works well. store this in the scratchpad first then add the MAC info then write to SD card.

Now for you boffins, when I have multiple ap's I get a response as follows from the lantronix

01-18-01-5E-46-F8, -58dBm, I , NONE, The_quick_brown_fox_jumped_over_
xx-xx-xx-xx-xx-xx, -xxdBm, x , xxxx, xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xx-xx-xx-xx-xx-xx, -xxdBm, x , xxxx, xxxxxxxxxxx
xx-xx-xx-xx-xx-xx, -xxdBm, x , xxxx, xxxxxxxxxxxxxxxxxxx

(assume x's other ap data)

This all comes down to speed I guess how would you cycle the AP data thru the scratchpad?

At the end of each AP I assume is a CR and NL which would be written to scratchpad? For Hserin's brilliance for writing directly to scratchpad there is no method to check each byte (that I can see) as it comes in.

 

[hippy]

That your "46" is corrupted suggests a baud rate problem of some kind, which I think you will have to try and resolve. I wouldn't be confident with any corruption in the data stream even if it were apparently predictable. That could always change in the future and turn all the work you do into wasted effort.

I'm also a bit concerned that you're missing the first two bytes of data. If there is a CR/LF at the end of the data, this can also be considered to be the start of the new data so I'd start serial capture with a qualifier for the CR/LF then grab N-bytes of the packet.

If you use receive with a CR/LF qualifier once you've got N-bytes you can then process the packet captured in scratchpad. the HSERIN command allows start of buffer to be defined so you can adjust that and restart HSERIN so it captures the next packet to a different area of scratchpad. You'll need to use background receive so it can wait for the CR/LF and capture the data in the background while you're getting on with processing the first.

I haven't used HSERIN so not sure what practical issues you'll run into. For example, does a background HSERIN check the qualifiers in the background, or does it block until those qualifiers are received ?

 

[jmumby]

I will have a look at by circuit to sort out the random -46- sometimes it appears ok, I have a resonator so I would have hoped my calibfreq days were over.

The first few bytes are being received (Im not using background receive at the moment) . It seems on the first query to the lantronix the first few bytes have garbage consecutive queries do not so I assumed they were missing. Lantronix bug

+-----+----------+-------+------------+-------+----------+-------+
|cr/lf| First Ap | Cr/lf | Second Ap  | Cr/lf | Third Ap | Cr/lf |
+-----+----------+-------+------------+-------+----------+-------+

If there is a CR/LF at the end of the data, this can also be considered to be the start of the new data so I'd start serial capture with a qualifier for the CR/LF then grab N-bytes of the packet.

Im trying to visualize what you are saying. Capture the data with a LF (from what I read the qualifier needs to be a single variable/constant) qualifier for a certain amount of bytes. Problem is the data length varies depending on SSID.

So I might abandon the SSID as it's not really relevant for the project. I will capture 35 bytes which will give me the top three AP's at least. I think I can sort by signal strength so it'll be the top three relevant AP's. I wonder if HSERIN would be quick enough for the following.

HSERIN 0,35,(10) 
HSERIN 35,35,(10)
HSERIN 70,35,(10)

I can't (from what I read) see anything about qualifiers for background receive.

 

[jmumby]

This might actually work, I have used the following

HSERIN [4000,READOUT2],0,40,(13)
HSERIN [4000,READOUT],41,40,(13)
HSERIN [4000,READOUT],81,40,(13)

But it is random in nature. I have three AP's here to test and sometimes it will pick up the last and first and sometimes the first and second and not all the chars come out correct. I think this is because the 28x1 is struggling with the 3.3v output from the lantronix. If I run the 28x1 at 5v it does not recognize the data from the lantronix at all!

I have shielded cables running about 100mm between the lantronix and the 28x1 so I hope it isn't just noise.

I may have to find a method to drive the output of the lantronix to 5v, it is 5volt tolerant.

 

[hippy]

The other alternative is to qualify on CR/LF then read as much as you can, parse what you get as a result and if you got three AP's all well and good, if you got less just deal with what you have.

The data you have in your first diagram ( post #8 ) is most likely sent back-to-back so the idea of restarting and reading separately ( post #9 ) for each part probably won't work; with a continuous bit stream the hardware will find it hard to re-synchronise to what are bytes. This is probably the corruption you are seeing.

 

[jmumby]

I seem to be having success with this

HSERIN [4000,READOUT2],0,40,(13)
HSERIN [4000,READOUT],41,40,(10)
HSERIN [4000,READOUT],82,40,(10)

why 13 for the first and 10 for the rest I have no idea. Anyhue I get a result as follows and I'm still battling this corrupt byte problem.

6A-46-4F-F8-AF‚BF, 01, -63 dBm, A, WEP,
00-18-4D-0E-46-F8, 11, -52 dBm, I, NONr,
01-18-01-5E-46-00, -58dBm, I , NONE,

Bytes 15,79 are always corrupt (, should be a - and r should be an E). I thought the scratchpad was stuffed so I wrote some code to write the whole thing to screen whilst filled with x's and it seems ok.

Also any one know the quickest way to flush the scratch pad?

Cheers