1
$\begingroup$

I have EPS/PDF figures which contain scatterplots consisting of a lot of dots resulting in files sizes from dozens to hundreds of MBytes. Now, many of the points are completely hidden, so I could remove them from the EPS/PDF.

The vector-graphics where generated in R (ggplot2 and generic R plot).

I am using GNU/Linux, is there a way to do this post generating the figures? Or do you know of some functionality in R?

And yes, I can export it to PNG/JPG/other, sure, but then my collaborator can not modify it without loss of quality (modification of text, etc.).

$\endgroup$
1
  • 1
    $\begingroup$ Sortof, not really the computer does not actually know whats visible or not. Its realtively easy for you to do it manually in a vector editor because you can know whats visible and whats not. Part of the reason why this is easy to do porgrammatically in the application is that the clipping mask is just discarded in final rasterisation. But its not feasible to do this at any other stage of the process as it would require a lot of extra work to account for conrer cases. And also its not 100% the same in all the cases. The best place to cull is before plotting. $\endgroup$
    – joojaa
    Jun 2, 2022 at 11:41

1 Answer 1

1
+100
$\begingroup$

This answer intends to flesh out joojaa's comment to a more complete answer.

It's possible. But probably not easy.

If you work with the EPS file, it's very likely to be ASCII or mostly ASCII so you can open it and edit with a text editor. Then you've got some obvious subtasks:

  1. locate the points in the file
  2. filter them somehow

To locate the points, it's all going to depend on exactly how the ggplot2 chooses to encode the points. They may be just decimal numbers or they may be something else. Level 2 PostScript added support for several kinds of binary encoding and binary tokens.

To filter the points. There are a few routes I can imagine, but I'm not sure which will come out easiest--or least challenging. You could modify the EPS program to not call any of the drawing operators, but just dump "surviving" points to a file or stdout with the = or print or writestring operators. Then, add a step which calls clip to determine if a point is "surviving" or maybe the infill would be useful for this. One issue is that in order "draw a point" in PostScript you really have to draw some other shape like a circle or rectangle. So when the program gets to the drawing phase, the point may no longer exist as just coordinates but a whole sequence of lines or curves which describe the actual shape being drawn. So far, all of these options involve modifying the PostScript code to steer or wrangle the points through a sieve instead of onto a display. If you get the code just right, then it should produce a small list of points which you can then edit back into the original EPS file.

Another option would be to write a much smaller PostScript program that just sets up the appropriate clip path (or just a regular path to use infill) and output the points. Then you could copy/paste the points out of the original, run them through this smaller, simpler sieve, and then paste them back in to the original.

Most of these steps could be done in some other, more convenient programming language assuming you can accomplish subtask 1 and find the points in the file to begin with.

In contrast, working with the PDF could provide some advantages, as well as some disadvantages. To start with, we're no longer in nice clean ASCII land. PDFs may contain a lot of just plain ASCII but there's a lot of binary as well (possibly a whole lot). PDF is not a programming language, just a flat data representation. But it's a flat data representation of printed output. So we may hit some of the same issues as working with PostScript, viz. the points may or may not exist in the file as just coordinates.

You might try passing the PDF through a tool like qpdf to get a readable ASCII file that can be edited and converted back to PDF.

After (some of) the above is done and working for a single file, you then will have the further task of automating the solution to run on multiple files. And depending on the details, that may make your process dependent on the details of gglot2's current behavior which may or may not be fully documented. At least if it depends upon the EPS being produced the same way forever into the future, that feels like a risky move.

I don't know much about R, but at the spot in the R code where you call ggplot2 to produce the output, like, you've got the points right there, right? I don't know what tools R has available for insideness testing, but for an axis-aligned box it ought to be pretty simple to code a function ( x>LL_x && x<UR_x && y>LL_y && y<UR_y ). If they're gone before they get to ggplot2, then it will have no opportunity to include them in the output.

Edit

The closest existing PS code that I know of was from a prototype program to visualize a Steinmetz solid. This was posted to usenet in 2012 (note: if you add custom CSS html-blob{white-space:pre}html-blob br{display:none}, you can view the original indentation in Google groups). It generates points with an O(n^3) algorithm but then saves the results in a file so it can alter the view without doing the generating step each time. So, it shows one way of passing points through a sieve (the function x y z f -> bool) and saving them in a separate file.

%!
% Point-field sampling
% with data caching (in a file),
% point-wise axial rotations,
% and perspective projection.

/fuzz 10000 def %the "grain" of eq
/oldeq /eq load def
/eq {
    fuzz mul round exch fuzz mul round oldeq
} def

/max {
    2 copy lt { exch } if pop
} def

%x y z  f  bool
/f {
    dup mul 3 1 roll    %z^2 x y
    dup mul 3 1 roll    %y^2 z^2 x
    dup mul 3 1 roll    %x^2 y^2 z^2
    2 index 2 index add %x^2 y^2 z^2  x^2+y^2
    3 index 2 index add %x^2 y^2 z^2  x^2+y^2  x^2+z^2
    3 index 3 index add %x^2 y^2 z^2  x^2+y^2  x^2+z^2  y^2+z^2
    max                 %x^2 y^2 z^2  x^2+y^2  max(x^2+z^2,y^2+z^2)
    max                 %x^2 y^2 z^2  max(x^2+y^2, max(x^2+z^2,y^2+z^2))
    4 1 roll pop pop pop %max(...)
    4 eq
} def


/filename (stein.pts) def

/low -2.2 def
/hi 2.2 def

%generate data by brute force, cache in file, plot
/pointfieldtocache {
    /res exch def
    /dt 1 res div def
    /fuzz res .5 mul def
        %fuzz affects the "closeness" of
        %the equality test. a lower fuzz will allow more
        %values to be equal.
        % 'res' gives thin lines
        % 'res .5 mul' gives wider "ribbons"

/outfile filename (w) file def
/outbuf 128 string def
    low dt hi {
        low dt hi {
            low dt hi { % xW yW zW "world" coords
                3 copy f {
                    3 copy 3 -1 1 { -1 roll
                        outbuf cvs outfile exch writestring
                        outfile (\n) writestring
                    } for %dump points to file
                    3 copy project
                    %2 copy exch = =
                    %2 copy transform exch = =  ()=
                    2 copy 2 copy moveto dt .5 mul 0 360 arc moveto
                    fill
                    /flushpage where {pop flushpage} if
                } if
                pop
            } for
            pop
        } for
        pop
    } for
outfile closefile
} def

%plot cached data from file
/pointfieldfromcache {
    /res exch def
    /dt 1 res div def
    /infile filename (r) file def
    /it 1 def
        %cvx exec
    %count 3 idiv {

    {
        {
            /it it 1 add def
            infile token not {stop} if % bail-out
            infile token not {stop} if % on any datafile issues
            infile token not {stop} if
                    %3 copy
                    project
                    2 copy 2 copy moveto dt .5 mul 0 360 arc moveto
            it res mod 0 oldeq {
                fill /flushpage where {pop flushpage} if
            } if
        } loop
    } stopped pop
    %} repeat
} def

/pointfield { %check if there's a readable data file
    { filename (r) file closefile } stopped not
    //pointfieldfromcache %yes! read it.
    { pop pop pointfieldtocache } ifelse %no! make one.
} def

% x y z ang -> x y' z'
/rotx {
    /theta exch def
    /z exch def
    /y exch def
    y theta cos mul
    z theta sin mul sub
    y theta sin mul
    z theta cos mul add
} def

% x y z ang -> x' y z'
/roty {
    /theta exch def
    /z exch def
    /y exch def
    /x exch def
    x theta cos mul
    z theta sin mul add
    y
    x theta sin mul neg
    z theta cos mul add
} def

% x y z ang -> x' y' z
/rotz {
    /theta exch def
    /z exch def
    /y exch def
    /x exch def
    x theta cos mul
    y theta sin mul sub
    x theta sin mul
    y theta cos mul add
    z
} def

% Eye coords
/ex .2 def %a little x-y skew adds "drama"
/ey .2 def
/ez 5 def

% x y z -> X Y
/project {
ang roty
ang .25 mul rotx
    /z exch def
    /y exch def
    /x exch def
    1 ez z sub div
    
x ez mul z ex mul sub
    1 index mul
    y ez mul z ey mul sub
    3 2 roll mul
} def

10 10 360 {
    /ang exch def
    %matrix currentmatrix
    300 400 translate
    100 100 scale
    60 pointfield
    %setmatrix fill
    /flushpage where {pop flushpage} if
    showpage
} for
$\endgroup$
2
  • $\begingroup$ Thanks much for your detailed answer luser droog! I actually was looking for a existing solution. I do not have the time to implement something on my own, though that would be interesting of course. $\endgroup$
    – Michael
    Jun 7, 2022 at 6:01
  • $\begingroup$ No problem. For posterity, I've added some code that does some of the things needed for subtask 2. PostScript is a programming language, so any Turing-computable computation can be done ... with effort. But for the larger task you describe, it really feels more profitable to attack the data higher up the food chain. Modifying the generated PS code is extra difficulty just to make something fragile (although sometimes results can be achieved with text transformation or simple changes to the program, so don't rule it out altogether). $\endgroup$
    – luser droog
    Jun 7, 2022 at 12:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.