Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options
 
PerlMonks  

Create email tracking image

by Bod (Priest)
on Mar 18, 2023 at 01:17 UTC ( #11151044=perlquestion: print w/replies, xml ) Need Help??

Bod has asked for the wisdom of the Perl Monks concerning the following question:

I'm looking for a neat way to create a 1x1 transparent PNG to track opening emails. The ways I have been doing it seem overly clumsy.

The code we use in our main CRM reads an image file and outputs it. This is c2012 code and my coding has improved considerably since then!

$file='incl/1x1transparent.png'; print "Content-type: image/png\n"; print "Set-Cookie: abc=xxx; SameSite=none; Max-Age=315576000\n" if $us +er; print "\n"; open IMG, $file; binmode IMG; while (sysread(IMG, $buffer,640)) { print $buffer; } close IMG; exit 0;

I wanted to avoid making an IO call to the filesystem to read in such a small file. My first attempt was to Base64 encode the image and serve that as text but that didn't work...

So I have come up with this solution:

print "Content-type: image/png\n\n"; binmode STDOUT; my $image = GD::Image->new(1, 1); my $white = $image->colorAllocate(255,255,255); $image->transparent($white); print $image->png; exit;

This is better but it seems a bit of overkill to use GD to do this.

Can you suggest a more elegant solution?

Replies are listed 'Best First'.
Re: Create email tracking image
by afoken (Chancellor) on Mar 18, 2023 at 12:16 UTC
    Can you suggest a more elegant solution?

    1. Don't. A sane mail client won't load remote images, so tracking won't work.

    2. Don't generate the same constant image over and over again. Use any tool you like to create the transparent pixel image as GIF or PNG once. I used Google ("transparent pixel gif") and found this page. It lists the resulting binaries conveniently as base64 strings. For a transparent pixel in GIF format, it is R0lGODlhAQABAIAAAP///wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==, in PNG, it is iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVQYV2NgYAAAAAMAAWgmWQ0AAAAASUVORK5CYII=. Now you can use MIME::Base64 or similar to convert that to a binary:

    use MIME::Base64 qw( decode_base64 ); # ... print "Content-type: image/gif\n\n"; binmode STDOUT; print decode_base64('R0lGODlhAQABAIAAAP///wAAACH5BAEAAAAALAAAAAABAAEAA +AICRAEAOw==');

    Of course, you could even get rid of MIME::Base64 by recoding that to hex, and use pack to write the binary without any extra modules:

    >perl -MMIME::Base64 -e 'print decode_base64("R0lGODlhAQABAIAAAP///wAA +ACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==")' > pixel.gif >perl -MMIME::Base64 -e 'print unpack("H*",decode_base64("R0lGODlhAQAB +AIAAAP///wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw=="))' 47494638396101000100800000ffffff00000021f90401000000002c00000000010001 +000002024401003b >perl -e 'print pack("H*","47494638396101000100800000ffffff00000021f90 +401000000002c00000000010001000002024401003b")' > pixel2.gif >md5sum pixel.gif pixel2.gif 325472601571f31e1bf00674c368d335 pixel.gif 325472601571f31e1bf00674c368d335 pixel2.gif >
    Like this:
    print "Content-type: image/gif\n\n"; binmode STDOUT; print pack('H*','47494638396101000100800000ffffff00000021f904010000000 +02c00000000010001000002024401003b');

    You could even get rid of pack:

    >perl -MMIME::Base64 -MData::Dumper -e '$x=decode_base64("R0lGODlhAQAB +AIAAAP///wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw=="); $Data::Dumper::Us +eqq=1; print Dumper($x);' $VAR1 = "GIF89a\1\0\1\0\200\0\0\377\377\377\0\0\0!\371\4\1\0\0\0\0,\0\ +0\0\0\1\0\1\0\0\2\2D\1\0;"; >

    Et voilą:

    print "Content-type: image/gif\n\n"; binmode STDOUT; print "GIF89a\1\0\1\0\200\0\0\377\377\377\0\0\0!\371\4\1\0\0\0\0,\0\0\ +0\0\1\0\1\0\0\2\2D\1\0;";

    (Note: That assumes an ASCII-based system. EBCDIC probably won't work with that constant.)

    And there is more: You can omit binmode on Unix, as Unix is 8-bit-clean. On DOS-based systems (DOS, OS/2, Windows), binmode prevents converting \n = \012 = 0x0A to \r\n = \015\012 = 0x0D, 0x0A. THIS GIF luckily contains no \n = 0x0A, so it does not matter if \n is converted or not. So for this constant, you can omit binmode also on DOS, OS/2, Windows:

    print "Content-type: image/gif\n\nGIF89a\1\0\1\0\200\0\0\377\377\377\0 +\0\0!\371\4\1\0\0\0\0,\0\0\0\0\1\0\1\0\0\2\2D\1\0;";

    Yipp, a single line of code works on Unix, DOS, OS/2, and Windows.

    Alexander

    --
    Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
      Don't. A sane mail client won't load remote images, so tracking won't work

      There is the difference between a programmer and a marketer!

      We get between 30% and 50% reported open rate from most emails on their first send1. So, at least 30% of our email lists have images turned on. Whilst it may seem an increase in privacy to block remote images, it actually means that the end user receives far less relevant content. A typical email campaign of ours splits the audience into four groups.

      1. Those who have not opened the email
      2. Those who have opened it but not acted on it
      3. Those who have partially acted (perhaps started watching a video)
      4. Those fully engaged (e.g. watch all the video)
      Each group will get different content and maybe that will be tailored depending on what they have engaged with in the past.

      Our email campaigns are quite sophisticated with some campaigns having over 100 different emails and variations for different people at different times. Most of the content is educational material designed to help our prospects, build our credibility and remind them that we exist. This level of sophistication and relevance would be limited without tracking images enabled as we would be relying solely on clicks and other triggers.

      1 Typically the first email in a campaign will be resent up to 5 times with a different subject to people who have not yet opened it

      2. Don't generate the same constant image over and over again.

      Yeah - that was exactly the point of the question...

      And there is more: You can omit binmode on Unix, as Unix is 8-bit-clean.

      Thank you kindly

      That is exactly the kind of solution I was looking for and having a mental block over!

Re: Create email tracking image
by LanX (Sage) on Mar 18, 2023 at 16:31 UTC
Re: Create email tracking image
by cavac (Vicar) on Mar 20, 2023 at 16:07 UTC

    There are many problems with those so called tracking bugs First of all, most email clients won't load remote content unless told to do so by the user (which is a per-message thing). Basically, you are 20 years too late.

    Secondly, spam filters look out for exactly those things and are very likely to flag your message as spam when they detect a remotely loaded 1 pixel image. In the course of mail processing and spam filtering, the mail system may already load the image (either for checking or for embedding in the message).

    Last but not least: Tracking bugs probably violate various privacy laws all around the world. The EU has the GDPR, and the UK has replaced that (after Brexit) with it's own version "UK-GDPR" (pretty much a word-for-word copy AFAIK). As you stated, you want to track user behaviour. This requires explicit consent by the end user!

    A GDPR violation can get expensive. I am not a lawyer, but this is what i can see:

    A user can state non-material damages (suffered stress due to your privacy violation or something similar) and claim compensation. GDPR Article 82 subsection 1:

    Any person who has suffered material or non-material damage as a resul +t of an infringement of this Regulation shall have the right to receive compensation from the controller or processor for the damage suffered.

    Tracking a user seems to (at least) violate the spirit of Article 25 "Data protection by design and default" of the GDPR, since there is no technical requirement to track the delivery of an email and no reasonable user expectation that the reading status of emails are tracked. Also, you need the other infrastructure in place to deal with data protection and handling of user requests pertaining to the GDPR (deletion requests, information requests, etc).

    Violating GDPR can result in administrative fines, see Article 83 subsection 4:

    Infringements of the following provisions shall, in accordance with pa +ragraph 2, be subject to administrative fines up to 10 000 000 EUR, or in the cas +e of an undertaking, up to 2 % of the total worldwide annual turnover of the p +receding financial year, whichever is higher: (a) the obligations of the controller and the processor pursuant to Ar +ticles 8, 11, 25 to 39 and 42 and 43; (b) the obligations of the certification body pursuant to Articles 42 +and 43; (c) the obligations of the monitoring body pursuant to Article 41(4).

    So technically, you could possibly get fined 10 million Euros (or more, depending on the financial success of your company) for adding a tracking mechanism to your outgoing email. Ooof, that doesn't sound like a project worth pursuing.

    PerlMonks XP is useless? Not anymore: XPD - Do more with your PerlMonks XP
      I seems to me it's a grey zone because too many of the big players are still profiting from user profiling.

      I just stumbled over a duckduckgo mail forwarding service promising to cleanse emails from tracking stuff.

      Apparently is (inter)national legislation not (yet) doing enough to force providers to automatically do so.

      Cheers Rolf
      (addicted to the 𐍀𐌴𐍂𐌻 Programming Language :)
      Wikisyntax for the Monastery

        I seems to me it's a grey zone because too many of the big players are still profiting from user profiling.

        Possibly. But it's one of those grey zones that are probably stricter for small companies than they are for bigger ones. Microsoft and Google can easily throw a few million bucks in the direction of their lawyers to make such problems magically go away (and earn those millions back in a matter of hours). I somehow doubt that Bod has the same options...

        PerlMonks XP is useless? Not anymore: XPD - Do more with your PerlMonks XP
      Basically, you are 20 years too late

      I know of no CRM or mail software that uses any other method...

      For example, we use Send In Blue for some transactional emails and use their email tracking. They add something like this to their emails:

      <img width="1" height="1" src="https://fgfcgfj.r.af.d.sendibt2.com/tr/ +op/ubVEi9KMORTJR7DtkZjC4DjoU--GFjX4L8Ive0tmrRHobDcJWwFSnv4PdLwO4VRiZ1 +6P1arvc-oAwLKIFN_U8zi7OcP99UZXHA7bwdO0_yAM3yvQPuk3ocgXu2pOprxjHTynTTj +cPblGap6rVGNweeM1Wro2dAoIK-xRqlhhQGGGN8" alt="" />
      (tracking code changed)

      Having a user click on a link is a more precise trigger of action as users and mail software can affect opening data but we still need to know who has definitely opened an email to be able to personalise relevant content. The data we collect in email processing is not personal data as defined by GDPR and our data processing systems are registered with the Information Commissioners Office as required here in the UK.

Re: Create email tracking image
by harangzsolt33 (Hermit) on Mar 19, 2023 at 03:15 UTC
    I use OEClassic email client software on Windows, and by default, it will not download any images. I have to click a link on top of each email if I want it to download and show all the images. In that case, most of the emails will not show anything anyway, because for some reason the https protocol is disabled in my email software. So, if the image url starts with https rather than http, it will never be downloaded. I am not sure how many people use this same software, but I just want to let you know that using a 1x1 pixel image is not a very reliable way to tell if the email was seen.

    If I wanted to know for sure if someone read my email, then I would write several paragraphs but I would only include part of the first sentence in the email to make the reader curious. Then I would include a link to click where they can read the REST of the email. If somebody clicks the link, then you can serve the page using a perl script, and you would automatically get an email-read notification that way! ;)

      because for some reason the https protocol is disabled in my email software

      I would hazard a guess that OEClassic hasn't been updated to use modern encryption APIs on Windows and the one it tries to use has been disabled by some windows security update.

      PerlMonks XP is useless? Not anymore: XPD - Do more with your PerlMonks XP
Re: Create email tracking image
by harangzsolt33 (Hermit) on Mar 19, 2023 at 02:59 UTC
    I don't know much about PNG files, but the smallest PNG file I was able to make was 127 bytes. That's a 1x1 pixel image. On the other hand, if your perl script responded with a 1x1 GIF image, it would only be 35 bytes. Much smaller! I can show you how to do that. I have to admit, I copied this code from someone else on here. lol :D

    #!/usr/bin/perl use strict; use warnings; SpitGIF(); exit; ################################################## # GIF | Graphics | v2023.2.5 # This function generates a single-pixel GIF of a given color # I took original code from PerlMonks and modified it a little bit. # Original code was written by a user named "turnstep" # and is called the World's Smallest GIF image. It produces a GIF # image that is only 35 bytes (or 43 bytes if it is transparent.) # Source: https://perlmonks.com/?node_id=7974 # # This function expects one or two arguments. The first argument # should be a hexadecimal number such as 0xCC00CC which would # produce a purple pixel. # # Note: This function will always generate the smallest possible # GIF image. If you would like to generate a 1x1 BMP image, # use the SpitBMP() function. # # Examples: # # SpitGIF() will send a black 1x1 pixel to the browser. # # SpitGIF(0xFFFFFF) will send a white 1x1 pixel to the browser # as response by printing it to stdout. # # SpitGIF(-1) will send a transparent pixel to the browser. # # The following example will simply return a grey 1x1 GIF # file's content in a string rather than printing it: # # my $GIF = SpitGIF(0x999999, '<STRING>'); # # Next example will save a red GIF pixel to a file called "red.gif" # # SpitGIF(0xFF0000, 'red.gif'); # # The next example will return a base64 inline code that can be # inserted into a HTML page and will produce a green GIF pixel: # # my $GIF = SpitGIF(0x00FF00, '<INLINE>'); # print "<IMG WIDTH=1 HEIGHT=1 BORDER=0 SRC='$GIF'>"; # ^ Here the $GIF # variable contains the GIF file's contents in base64 encoding # rather than a gif file name. Newer web browsers will recognize # this and will render the image correctly. Very old web browsers # such as MSIE6 will just display a missing image icon. # # Usage: SpitGIF(COLOR, [OUTPUT]) # sub SpitGIF { my $COLOR = defined $_[0] ? $_[0] : 0; my $GHOST = ($COLOR & 0xFF000000) ? "!\xF9\4\5\x10\0\0\0" : ''; $COLOR = substr(pack('N', $COLOR & 0xffffff), 1, 3); # Create the world's smallest GIF image: my $GIF = "GIF89a\1\0\1\0\x90\0\0$COLOR" . "\0\0\0$GHOST,\0\0\0\0\1\0\1\0\0\2\2\4\1\0;"; # If $_[1] is not empty, then... unless (defined $_[1] && length($_[1]) && $_[1] =~ m/\S+/) { binmode(STDOUT); return print "Content-Type: image/gif\nContent-Length: ", length($GIF), "\n\n", $GIF; } my $F = $_[1]; if ($F eq '<STRING>') { return $GIF; } if ($F eq '<INLINE>') { EncodeBase64($GIF, 1); return "data:image/gif;base64,$GIF"; } print "\nSaving 1x1 GIF image to $F\n"; return CreateFile($F, $GIF); } ################################################## # String | v2022.11.4 # This function converts a string to text using the # standard Base64 encoding algorithm. # # THIS FUNCTION CHANGES THE VALUE OF THE FIRST ARGUMENT! # # The first argument should be a string or a string reference. # The second argument MAY contain a custom 65-byte # character set. The 65th byte will be used for padding. # # The second argument MAY also be 0 or 1: # 0 = Use standard Base64 encoding which ends with +/ (default) # 1 = Use web-safe Base64 encoding which ends with -_ # # Usage: EncodeBase64(STRING, [CHARSET]) # sub EncodeBase64 { defined $_[0] or return $_[0] = ''; # Prepare character set. my $PADDING = '='; my $BASE64 = defined $_[1] ? $_[1] : 0; if (length($BASE64) >= 64) { $PADDING = substr($BASE64, 64, 1); } else { $BASE64 = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789' . (($BASE64 eq '1') ? '-_' : '+/'); } # Get string reference. my $REF = ref($_[0]); if (length($REF)) { $REF eq 'SCALAR' or return 0; $REF = $_[0]; defined $$REF or return $$REF = ''; } else { $REF = \$_[0]; } my ($i, $OUT, $L) = (0, '', length($$REF)); while ($i < $L) { # Read 3 bytes. my $A = vec($$REF, $i++, 8); my $B = vec($$REF, $i++, 8); my $C = vec($$REF, $i++, 8); # Write 4 bytes. $OUT .= substr($BASE64, $A >> 2, 1); $OUT .= substr($BASE64, (($A & 3) << 4) | ($B >> 4), 1); $OUT .= substr($BASE64, (($B & 15) << 2) | (($C >> 6) & 3), 1); $OUT .= substr($BASE64, $C & 63, 1); } # Replace last couple of bytes with padding. my $DIFF = $i - $L; substr($OUT, length($OUT) - $DIFF, $DIFF) = $PADDING x $DIFF; $$REF = $OUT; # Replace original string with the new one. undef $OUT; # Free up memory. return 1; } ################################################## # File | v2022.11.8 # Creates and overwrites a file in binary mode. # If the file has already existed, it erases the # old content and replaces it with the new content. # Returns 1 on success or 0 if something went wrong. # # Usage: STATUS = CreateFile(FILENAME, CONTENT) # sub CreateFile { my $F = defined $_[0] ? shift : ''; $F =~ tr`<>*%$?\x00-\x1F\"\|``d; # Remove illegal characters. local *FILE; open(FILE, ">$F") or return 0; binmode FILE; foreach (@_) { defined $_ and length($_) and print FILE $_; } close FILE; -e $F or return 0; # File exists? -f $F or return 0; # It's a plain file? return 1; } ################################################## # BMP | v2022.10.11 # This function returns a 1x1 truecolor BMP image # with a Version 40 header which is considered standard. # All photo editors and web browsers should be able # to open this type of file with no problem. # (The entire BMP image is 58 bytes long.) # # The first argument (RGBColor) should be a 32-bit integer # which holds the R-G-B values in the following order: 0x00rrggbb # # Usage: STRING = SpitBMPv40(RGBColor) # # Example: SpitBMPv40(0xff0000) -- returns a red pixel # SpitBMPv40(0x00ff00) -- returns a green pixel # SpitBMPv40(0x0000ff) -- returns a blue pixel # sub SpitBMPv40 { my $PIXEL = defined $_[0] ? $_[0] & 0xffffff : 0; #################################################################### +############################ # v40 Standard Windows BMP Structure # # BMP Version Bits Per Pixel + Number of Colors Used (0=All) # Header Length \ Width | Compressi +on / Important Colors (0=All) # File Size \ \ \ Height | / DataSiz +e / / # \ \ \ \ \ | / / + / / return 'BM' . pack('V6vvV7', 58, 0, 54, 40, 1, 1, 1, 24, 0, 4, 0, 0, + 0, 0, $PIXEL); # | | | \/ # File Signature Reserved Planes Recommen +ded Horizontal & Vertical Density for Printing } ##################################################
ChatGPT's solution (was: Re: Create email tracking image)
by Bod (Priest) on Mar 21, 2023 at 22:56 UTC

    Just for fun I asked ChatGPT how do do this using the prompt: "What is the best way to create a 1x1 transparent image in Perl for use in email tracking?"

    It came up with this solution:

    use GD; # Create a new transparent image my $image = GD::Image->newTrueColor(1, 1); $image->alphaBlending(0); $image->saveAlpha(1); my $transparent = $image->colorAllocateAlpha(0, 0, 0, 127); $image->transparent($transparent); # Output the image print "Content-Type: image/png\n\n"; print $image->png;
    And then went on to explain how it worked...

    I prefer the solution put forward by afoken in Re: Create email tracking image

Re: Create email tracking image
by Anonymous Monk on Mar 18, 2023 at 09:58 UTC

      That doesn't help...
      Not only is Return-receipt-to not universally supported, but where it is supported, it returns an email. If we send out 2000 emails, we don't want 1000 or so emails back that we need to scrape to write to our database that the contact has viewed the email - chance the reason for doing it with an invisible image that gets downloaded.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://11151044]
Approved by kcott
Front-paged by kcott
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others scrutinizing the Monastery: (2)
As of 2023-03-22 03:29 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Which type of climate do you prefer to live in?






    Results (60 votes). Check out past polls.

    Notices?