Reading information from file

[VanGog]

I would like to continue in the thread, where Anthony wrote his post about reading meta information from file, viewtopic.php?f=1&t=21124#p85782 in Canvas and offset. I think about using BW image as a less space consuming storage, bearing information about files structure, and probably also on the same basis possibility to save information about missing files.

My basic question is whether my idea is realizable, for known issues of slow txt:- command, and whether is it possible to list coordinates of missing pixels.

I would like to create binary mask. Only white pixels could be needed. What I want to do, is a grid like a chessboard, which bear "names" of images existing in my directory. These image I am going to join together into bigger images. The binary mask tells to a program, where starts one group of images, and where ends. I could do it in two colors, e.g. red and white. Because if there would be 100x100 pixels, divided by 10x10 squares (red,white,red,white,red,white,red,white,red,white), so it has bigger size than (empty,white,empty,white,empty,white,empty,white,empty,white). But now comes my question. If the area is empty, it does not mean, there is not information. It is also information of file names. For example, image position
17480,11032 refers to image 17480_11032, image position 17481,11032 refers to image 17481_11032, 17480,11033 refers to image 17480_11033, image position 17481,11033 refers to image 17481_11033, this is first quadrant, which is empty .

I tried this example, but with errors:

Code: Select all

convert -size 100x100 -page 17580x11132+17480x11032 xc:none ^
-stroke white ^
-draw "point 17482,11034" -draw "point 17483,11034" ^
-draw "point 17482,11035" -draw "point 17483,11035" ^
metadata.gif

I created the image, but here I stuck the txt:- problem because it tried to print all pixels:

Code: Select all

convert -size 100x100 -page 17580x11132+17480x11032 xc:none ^
-stroke white ^
-draw "point 17482,11034" -draw "point 17483,11034" ^
-draw "point 17482,11035" -draw "point 17483,11035" ^
txt:-

14,1: ( 0, 0, 0, 0) #0000000000000000 none
15,1: ( 0, 0, 0, 0) #0000000000000000 none

I would need to print only the pixels in the image size starting from 17480x11032.
It is absurd, that it tries to print whole image even it almost contains no pixels. Can you help?

I have more scenarios:
1) to print only empty pixels inside the image size (100x100 pixels)
- I need only coordinates
- no color information, that could speed it up
2) to print only white pixels inside the image size (100x100 pixels)
- I need only coordinates)
- no color information, that could speed it up
3) to all pixels in image size (100x100 pixels)
- I need only coordinates
- I need to recognize whether it is empty or white, but I would like to shorten the output as possible to speed up the pass into CMD.

I am fascinated by this benefit from IM, the fact, that image can bear information like a database to be used by external program.

[anthony]

I am not quite following this. But you can not have two 'x' in a geometry expression. IM would probably accept it, but it could just as easilly confuse it.

A geometry expression has the form WxH+X+Y where the capitals are numbers, 'x' is an 'x' and '+' could be plus or minus. You can also have other characters which are treated simply as an option flags, %@><! if the operator understands it!

See IM Examples, Basics, Geometry Arguments
http://www.imagemagick.org/Usage/basics/#arg_geometry

[VanGog]

Do I have it right now?

Code: Select all

convert -size 100x100 -page 100x100+17480+11032 xc:none ^
-stroke white ^
-draw "point 17482,11034" -draw "point 17483,11034" ^
-draw "point 17482,11035" -draw "point 17483,11035" ^
map.gif

map.gif GIF 100x100 17580x11132+17480+11032 8-bit PseudoClass 2c 156B 0.000 u 0:00.000

It should be 100x100 px image, working area should be started at point
17480,11032 and ending at point
17580,11132

There should be four white pixels, but I probably have one more error that I cannot find them in the image.

PS: What does it mean "special 'flag' characters "? What is it good for?
PS2: So can I print information about coordinates from the working area? The area of 100x100 pixels. If I use txt:- so all pixels are printed even those in the offset. And that takes too long to print.

[anthony]

Some operator has 'special characters' or 'flags' in the 'geometry' argument

• '%' often means the size is a percentage of the original image,
• for resize '!' means ignore aspect ratio in calculation, and just use dimensions as is for the resized image.
• In Crop '!' means a viewport crop, which changes the 'page offset' of the result to be relative to be relative to the geometry given.

And so on.

Image creation. You have a large offset very very small virtual canvas.
Hmmm Draw does NOT understand virtual offsets either!

For your Draw... Points are drawn using 'FILL' color, not stroke color. Their is also two types of 'point' drawing
(not counting very small circles)

you can draw a rectangle of 4 pixels instead of each individual point. 'fill' is used for insides (the default) 'stroke' for the outline (notmally set to 'transparency' so it is not seen by default.

This works!
convert -size 200x200 -page 300x300+100+100 xc:white -fill red \
-draw "point 111,111" -draw "point 111,112" \
-draw "point 112,111" -draw "point 112,112" t.gif

See IM examples, Draw
http://www.imagemagick.org/Usage/draw/

Also see Drawing on animations (layer images)
http://www.imagemagick.org/Usage/anim_mods/#draw

[VanGog]

Anthony, thank you very much. I will read carefuly.

[VanGog]

It seems to me that better solution than using huge offset would be insert property into image saying, what numbers are assigned to the first pixel. The image size will be without offset having for example 100x100.

I would create png, where is a background and foreground layer.

There are many little image files in my directory, and they should be ordered into groups which will created bigger images. Here the comparison to chessboard, where one chessboard field is one completed image. However I need to check if all files are there.

I would read directory for files, and if the file doesn't exist, I would create red pixel on his location in the image layer.

Now, if the completed image has odd number, then it should be empty area in the layer. But if the image is even number, then the area of pixels is white color.

Only question here is, should I create one layer for white and one layer for red, or should I write white and red together? This is for reading. I would like to check missing files later, I would want to read and print the red pixels only.

[anthony]

It seems to me that better solution than using huge offset would be insert property into image saying, what numbers are assigned to the first pixel. The image size will be without offset having for example 100x100.

Actually that is what exactly happens! Virtual Canvas size, and the offset on that canvas are just a property of the image.
PNG saves these (offset offically, size unofficially via IM specific profile). GIF saves these too, but only positive offsets. JPG does not save this property.

The problem is some operators do not understand 'virtual canvases', or use a different system for more flexibility, typicaly the very low level ones Draw and Composition are the main ones.

I would create png, where is a background and foreground layer.

There are many little image files in my directory, and they should be ordered into groups which will created bigger images. Here the comparison to chessboard, where one chessboard field is one completed image. However I need to check if all files are there.

I would read directory for files, and if the file doesn't exist, I would create red pixel on his location in the image layer.

Create a Big canvas 'red' in color, then read and 'layer' each image onto that canvas.

Sounds like something perfect for IM v7 using a shell controlling a IM sub-process (co-process)
I scripted this up as an example -- but something is not going right -- at least not yet. (it is alpha development)

However a shell creating a long IM command should work fine for this.
Something like the 'animation scripts' exampled in
http://www.imagemagick.org/Usage/warping/#animations

Now, if the completed image has odd number, then it should be empty area in the layer. But if the image is even number, then the area of pixels is white color.

Pardon that does not make sense to me.


Only question here is, should I create one layer for white and one layer for red, or should I write white and red together? This is for reading. I would like to check missing files later, I would want to read and print the red pixels only.[/quote]

[VanGog]

Create a Big canvas 'red' in color, then read and 'layer' each image onto that canvas.

Why? I am not going to join the images now. This script should be for storing information about images in the directory. As prepare step before I would run new script that will join the images. This is the reason why I want to skip odd images, to make the file less size. Also one image here means one pixel. One pixel bears two informations:
1) is the image named with odd number? Or with even number?
2) does the image exist or not?

Sounds like something perfect for IM v7 using a shell controlling a IM sub-process (co-process)
I scripted this up as an example -- but something is not going right -- at least not yet. (it is alpha development)

However a shell creating a long IM command should work fine for this.
Something like the 'animation scripts' exampled in
http://www.imagemagick.org/Usage/warping/#animations

Now I don't understand how animations relates to this topic. Do you say, that animations will use some technology how to use with big count of arguments?

This is would be also subject of new topic later. Because I will need to join all the images and they are very many.

[anthony]

Create a Big canvas 'red' in color, then read and 'layer' each image onto that canvas.

Why? I am not going to join the images now. This script should be for storing information about images in the directory. As prepare step before I would run new script that will join the images. This is the reason why I want to skip odd images, to make the file less size. Also one image here means one pixel. One pixel bears two informations:
1) is the image named with odd number? Or with even number?
2) does the image exist or not?

Okay, My misunderstanding. The image with red background is just a map of images available.
Sounds like a shell script creating a 'Enumerated Pixel Image' (TXT suffix)
http://www.imagemagick.org/Usage/files/#txt
You create a header, plus one line for each image present. You can set background color to red when reading this into IM for conversion to a proper image file format. Then red will be used or any pixel not defined. Each line will then be white or transparent for any existing image, depending on the number.

How would you equate the image number to pixel, in the 100x100 image?
Can you give an example list of the filenames involved? (for the number format in the name and image suffix that is appended)

Sounds like something perfect for IM v7 using a shell controlling a IM sub-process (co-process)
I scripted this up as an example -- but something is not going right -- at least not yet. (it is alpha development)

However a shell creating a long IM command should work fine for this.
Something like the 'animation scripts' exampled in
http://www.imagemagick.org/Usage/warping/#animations

Now I don't understand how animations relates to this topic. Do you say, that animations will use some technology how to use with big count of arguments?

Apologies if that was not clear.

Animations and layered use exactly the same image handling techniques. They both represent a list of images, with an offset on a virtual canvas. animations just display them in sequence, while layers display them like a stack of overhead transparency.

The link reference about drawing on images with an offset just happened to be for an animation, but applies equally well to a sequence of layer images. EG: drawing on a set of images with different offsets.

[VanGog]

Here my original idea for the grid:
http://s17.postimage.org/as62e3uu7/completed_grid_2.png


The first chessboard represents zoomed square of little images that should be joined together - in this order as you see them. Here the image represents 10x10px.
The name of images are starting from source_566233_357032.jpg and ending with source_566242_357041.jpg. They are joined into one image called 566233_357032 (which is target image)... we could call it also target_566233_357032.png. The name is derivatived from upper left source image.

The second "chessboard" is similar organization, but this time you see all images, organized into one bigger image. The image represents 100x100px.
Note: second chessboard will be joined later, when color corrections and resize will be done to the completed images.

[VanGog]

You came with different design.

Your design is more simple in the idea of reading files line by line (one line means one image), but also does not offer the feature of grid based naming of files.

According my scheme, you need to know starting "offset" values (not offset at all, but real numbers of the first image). Than you can calculate correct name in the image.

In the real the final image will have either 1024x1024 or 4096x4096px.

[anthony]

If the offsets are in the names and not as a offset in the image (difficult as offsets may have integer size limits that are different to images. GIF for example I believe is limited to 16 bit. Not certain abot PNG), then IM will have the numbers as separate values to use. to position images in the final layer image.

But if the numbers are actually part of the filename, then IM can't extract the numbers, (IMv7 may be able to but not in the immediate future). You will need to extract the numbers from filename using shell/awk/perl string processing functions.

Now as you are generating a map of one pixel per image, then Enumerated Text file format (TXT:) is still probably still the right way to go, though it is such as simple format that it does not have an understanding of virtual canvas offsets.

Okay, shell script time. (assuming you already determined the 'offset' and 'size' of the map image.
This uses a special bash "<<<" operator to "read" multiple columns from a string (the filename)

Code: Select all

#!/bin/bash
off_x=566233
off_y=357032
size_x=1024
size_y=1024

( echo "# ImageMagick pixel enumeration: $size_x,$size_y,256,RGB"
  for file in source_*.jpg
  do
     IFS='_.' read base x y suffix <<< "$filename"
     echo >&2 "Processing:  $base  $x  $y  $suffix"   # debug check
     x=$(( x - off_x ))
     y=$(( y - off_y ))
    echo "$x,$y: (255,256,256)   # white"
  done
) | convert -page +$off_x+$off_y -background black txt:-  map.png

This should generate a black image (for any pixel that was NOT specified in the loop), with a white pixel for every location in which a filename is found (specificed in the loop). The image is given the offset that was subtracted from the coordinates that was plotted.

Now if you underlay a "pattern:gray50" with the right coloring (red and black), you can then highlight the 'missing' tiles.



The alternative is to loop over every pixel, skipping when x+y == even_number, and checking if the next filename (number sorted) matches, red is not and skip to next pixel.

That scheme does not need to involve images at all!

[VanGog]

Thanks for the code. It looks really simple and no problem to do it in batch. Also, your command to assign values to variables from filename string is very similar to list (var1,var2,var3) = explode(delimiter,filename); command in PHP. I will test your command when the time will come to work with IM, because now I am occupied with programming.

[anthony]

This is 'here string' syntax, as it is called is purely BASH, and probably will not work for any other shell. There is a number of other methods, but they are typically more complex, or uses multiple commands.

Fred's scripts for example use a more primitive, extract one value at a time, method as the BASH on MacOSX is not always 'new' enough.

Also see
http://www.ict.griffith.edu.au/anthony/ ... ript.hints
and search the plain text file for...
"Break string into words"
and the next section
"Results from commands