Jq stuff: Difference between revisions

From SoftwareGuy
Jump to navigation Jump to search
Mark (talk | contribs)
Mark (talk | contribs)
 
(9 intermediate revisions by the same user not shown)
Line 9: Line 9:
Fancy way to find the scanId of a selected system via bash and jq
Fancy way to find the scanId of a selected system via bash and jq


<code>if [ -z $1 ]; then whichIE="uvx"; else whichIE=$1; fi</code>
<pre>if [ -z $1 ]; then whichIE="uvx"; else whichIE=$1; fi
printf ".lastVehicleRecord.scans[] | select(.type | endswith(\""$whichIE"\"))\n" > parms.jq


<code>printf ".lastVehicleRecord.scans[] | select(.type | endswith(\""$whichIE"\"))\n" > parms.jq</code>
wget -qO heartbeat <nowiki>http://$ip/heartbeat</nowiki>
cnt=$(jq '.lastVehicleRecord.scans | length' heartbeat)


<code>wget -qO heartbeat <nowiki>http://$ip/heartbeat</nowiki></code>
id=$(jq -f parms.jq heartbeat | jq '.id')
if [ ! -z $id ]; then
    wget -qO $id.orig.tiff <nowiki>http://$ip/requestImage/?scanId=$id</nowiki>
fi</pre>More interesting jq examples:


<code>cnt=$(jq '.lastVehicleRecord.scans | length' heartbeat)</code>
<code>"plateCoordinates":[898, 210, 144, 44] "plateCoordinatesRelative":[898, 210, 144, 44],</code>


<code>id=$(jq -f parms.jq heartbeat | jq '.id')</code>
<code>jq ".plateCoordinates | .[1]" upload_test/2023-05-12_12:08:07.txt</code>


<code>if [ ! -z $id ]; then</code>
== Tiff info parsing ==
Parsing examples of .tiff files extracing meta data from the headers.


<code>    wget -qO $id.orig.tiff <nowiki>http://$ip/requestImage/?scanId=$id</nowiki></code>
<code>tiffinfo 2023-05-07\ 13_57_17_evx_left_3845.tiff | grep 65001 | sed 's/  Tag 65001: //' | sed 's/\\//'g | jq</code>


<code>fi</code>
<code>tiffinfo 2023-05-07\ 13_57_17_evx_left_3845.tiff | grep 65001 | sed 's/  Tag 65001: //' | sed 's/\\//'g | jq '.timestamp'</code>
 
<code>tiffinfo 2023-05-07\ 13_57_17_evx_left_3845.tiff | grep 65001 | sed 's/  Tag 65001: //' | sed 's/\\//'g | jq '.renderData.type'</code>
 
Remove stderr from stream
 
<code>tiffinfo 189.tiff 2>/dev/null | grep 65001 | sed 's/  Tag 65001: //' | sed 's/\\//'g | jq '.renderData.type'</code>
 
== LPR parsing (Axis P1455) ==
The camera should be set to "save full frame".
 
Parsing the json from the axis camera:
 
<code>jq ".ImageArray[0].BinaryImage" /home4/Events/upload_test/2023-06-15_15:27:01.txt > image.b64</code>
 
Next remove the quotes in the image.b64 file, or:
 
<code>jq --raw-output ".ImageArray[0].BinaryImage" /home4/Events/upload_test/2023-06-15_15:27:01.txt > image.b64</code>
 
Then:
 
<code>base64 -d image.b64 > image.jpg</code>
 
Now to get the lpr image:
 
<code>jq ".imagesURI[0]"  /home4/Events/upload_test/2023-06-15_15:27:01.txt</code>
 
then use that url to pull the license plate image.  You may have to trim the returned url to just include "<code>getImage&name=3</code>", not the full url...
 
Alternatively you could just grab the plate coordinates from the json and clip the image, presumably.
 
<code>jq ".plateCoordinates"  /home4/Events/upload_test/2023-06-15_15:27:01.txt</code>
 
== Service Log parsing ==
As of July 25, 2023, the services logs have issues with quotes and backslashes.
 
Basic parsing of the log is done like this:
 
<code>jq ".[0].message" getServiceLogs.json</code>
 
provides just the first message element.
 
I edit the file and remove the quote around the contents of the "message" element.  (in emacs macro search for message, space forward to the starting quote, now search for }" then search for dateLong and end macro.  Next I am removing the /, both of these in emacs.
 
A bunch of sed stuff to help with the parsing...
 
<code>sed 's/\\\"\\\"/""""/g' getServiceLogs.json | sed 's/\\//g' | sed 's/\}"/\}/g' | sed 's/""/"/g' | sed 's/\"\[{/\[{/g' | sed 's/}\]\"/}\]/g' | sed 's/\"{/{/g' > getServiceLogs2.json</code>

Latest revision as of 15:55, 26 July 2023

Using jq[edit | edit source]

Parsing

Command to return the id number of the last scan:

cnt=$(jq '.lastVehicleRecord.scans | length' heartbeat)

Fancy way to find the scanId of a selected system via bash and jq

if [ -z $1 ]; then whichIE="uvx"; else whichIE=$1; fi
printf ".lastVehicleRecord.scans[] | select(.type | endswith(\""$whichIE"\"))\n" > parms.jq

wget -qO heartbeat http://$ip/heartbeat
cnt=$(jq '.lastVehicleRecord.scans | length' heartbeat)

id=$(jq -f parms.jq heartbeat | jq '.id')
if [ ! -z $id ]; then
    wget -qO $id.orig.tiff http://$ip/requestImage/?scanId=$id
fi

More interesting jq examples:

"plateCoordinates":[898, 210, 144, 44] "plateCoordinatesRelative":[898, 210, 144, 44],

jq ".plateCoordinates | .[1]" upload_test/2023-05-12_12:08:07.txt

Tiff info parsing[edit | edit source]

Parsing examples of .tiff files extracing meta data from the headers.

tiffinfo 2023-05-07\ 13_57_17_evx_left_3845.tiff | grep 65001 | sed 's/  Tag 65001: //' | sed 's/\\//'g | jq

tiffinfo 2023-05-07\ 13_57_17_evx_left_3845.tiff | grep 65001 | sed 's/  Tag 65001: //' | sed 's/\\//'g | jq '.timestamp'

tiffinfo 2023-05-07\ 13_57_17_evx_left_3845.tiff | grep 65001 | sed 's/  Tag 65001: //' | sed 's/\\//'g | jq '.renderData.type'

Remove stderr from stream

tiffinfo 189.tiff 2>/dev/null | grep 65001 | sed 's/  Tag 65001: //' | sed 's/\\//'g | jq '.renderData.type'

LPR parsing (Axis P1455)[edit | edit source]

The camera should be set to "save full frame".

Parsing the json from the axis camera:

jq ".ImageArray[0].BinaryImage" /home4/Events/upload_test/2023-06-15_15:27:01.txt > image.b64

Next remove the quotes in the image.b64 file, or:

jq --raw-output ".ImageArray[0].BinaryImage" /home4/Events/upload_test/2023-06-15_15:27:01.txt > image.b64

Then:

base64 -d image.b64 > image.jpg

Now to get the lpr image:

jq ".imagesURI[0]"  /home4/Events/upload_test/2023-06-15_15:27:01.txt

then use that url to pull the license plate image. You may have to trim the returned url to just include "getImage&name=3", not the full url...

Alternatively you could just grab the plate coordinates from the json and clip the image, presumably.

jq ".plateCoordinates"  /home4/Events/upload_test/2023-06-15_15:27:01.txt

Service Log parsing[edit | edit source]

As of July 25, 2023, the services logs have issues with quotes and backslashes.

Basic parsing of the log is done like this:

jq ".[0].message" getServiceLogs.json

provides just the first message element.

I edit the file and remove the quote around the contents of the "message" element. (in emacs macro search for message, space forward to the starting quote, now search for }" then search for dateLong and end macro. Next I am removing the /, both of these in emacs.

A bunch of sed stuff to help with the parsing...

sed 's/\\\"\\\"/""""/g' getServiceLogs.json | sed 's/\\//g' | sed 's/\}"/\}/g' | sed 's/""/"/g' | sed 's/\"\[{/\[{/g' | sed 's/}\]\"/}\]/g' | sed 's/\"{/{/g' > getServiceLogs2.json