- python 2017-03-19T16:58:15+01:00 https://fadeit.dk/blog/tag/python.html Python logging handler for Mailgun 2015-10-12T13:25:30+02:00 https://fadeit.dk/blog /2015/10/12/mailgun-python-log-handler <p>Yet another reason to love python is the wonderful logging API provided by a standard library. Having a decent logger provided by the language saves developers time from integrating with some third-party library, but most importantly it contributes to a clean system where all modules use the same logger. Out of the box, there is a <a href="https://docs.python.org/3/library/logging.handlers.html">handler</a> for almost any typical use-case, but the best feature is the option to create your own with relative ease. So-far we’ve been using <a href="https://docs.python.org/3/library/logging.handlers.html#smtphandler">SMTPHandler</a>, but since moving to <a href="https://www.mailgun.com/">Mailgun</a>, I decided to experiment with a custom handler to submit logs via their API instead. I’d like to mention that creating a custom logging handler is not a requirement, it is still possible to use their SMTP server with SMTP handler. To get started, we need to extend Handler baseclass and override emit method to make a POST request.</p> <div class="language-python highlighter-rouge"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">logging</span> <span class="kn">import</span> <span class="nn">requests</span> <span class="k">class</span> <span class="nc">MailgunHandler</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">Handler</span><span class="p">):</span> <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">api_url</span><span class="p">,</span> <span class="n">api_key</span><span class="p">,</span> <span class="n">sender</span><span class="p">,</span> <span class="n">recipients</span><span class="p">,</span> <span class="n">subject</span><span class="p">):</span> <span class="c"># run the regular Handler __init__</span> <span class="n">logging</span><span class="o">.</span><span class="n">Handler</span><span class="o">.</span><span class="n">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="bp">self</span><span class="o">.</span><span class="n">api_url</span> <span class="o">=</span> <span class="n">api_url</span> <span class="bp">self</span><span class="o">.</span><span class="n">api_key</span> <span class="o">=</span> <span class="n">api_key</span> <span class="bp">self</span><span class="o">.</span><span class="n">sender</span> <span class="o">=</span> <span class="n">sender</span> <span class="bp">self</span><span class="o">.</span><span class="n">recipients</span> <span class="o">=</span> <span class="n">recipients</span> <span class="bp">self</span><span class="o">.</span><span class="n">subject</span> <span class="o">=</span> <span class="n">subject</span> <span class="k">def</span> <span class="nf">emit</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">record</span><span class="p">):</span> <span class="c"># record.message is the log message</span> <span class="k">for</span> <span class="n">recipient</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">recipients</span><span class="p">:</span> <span class="n">data</span> <span class="o">=</span> <span class="p">{</span> <span class="s">"from"</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">sender</span><span class="p">,</span> <span class="s">"to"</span><span class="p">:</span> <span class="n">recipient</span><span class="p">,</span> <span class="s">"subject"</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">subject</span><span class="p">,</span> <span class="s">"text"</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">record</span><span class="p">)</span> <span class="p">}</span> <span class="n">requests</span><span class="o">.</span><span class="n">post</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">api_url</span><span class="p">,</span> <span class="n">auth</span><span class="o">=</span><span class="p">(</span><span class="s">"api"</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">api_key</span><span class="p">),</span> <span class="n">data</span><span class="o">=</span><span class="n">data</span><span class="p">)</span> </code></pre> </div> <p>I have created a simple flask application that registers the handler and also implemented a route that triggers an error to test it out.</p> <div class="language-python highlighter-rouge"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">logging</span> <span class="kn">from</span> <span class="nn">flask</span> <span class="kn">import</span> <span class="n">Flask</span> <span class="kn">from</span> <span class="nn">mailgun_handler</span> <span class="kn">import</span> <span class="n">MailgunHandler</span> <span class="n">app</span> <span class="o">=</span> <span class="n">Flask</span><span class="p">(</span><span class="n">__name__</span><span class="p">)</span> <span class="c"># Setup logging handler</span> <span class="n">mail_handler</span> <span class="o">=</span> <span class="n">MailgunHandler</span><span class="p">(</span> <span class="n">api_url</span><span class="o">=</span><span class="s">'https://api.mailgun.net/v3/fadeit.dk/messages'</span><span class="p">,</span> <span class="n">api_key</span><span class="o">=</span><span class="s">'secret'</span><span class="p">,</span> <span class="n">sender</span><span class="o">=</span><span class="s">'noreply@fadeit.dk'</span><span class="p">,</span> <span class="n">recipients</span><span class="o">=</span><span class="p">[</span><span class="s">'ss@fadeit.dk'</span><span class="p">,</span> <span class="s">'ja@fadeit.dk'</span><span class="p">],</span> <span class="n">subject</span><span class="o">=</span><span class="s">'Application error!'</span> <span class="p">)</span> <span class="n">mail_handler</span><span class="o">.</span><span class="n">setLevel</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">ERROR</span><span class="p">)</span> <span class="n">mail_handler</span><span class="o">.</span><span class="n">setFormatter</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">Formatter</span><span class="p">(</span><span class="s">''' Message type: </span><span class="si">%(levelname)</span><span class="s">s Location: </span><span class="si">%(pathname)</span><span class="s">s:</span><span class="si">%(lineno)</span><span class="s">d Module: </span><span class="si">%(module)</span><span class="s">s Function: </span><span class="si">%(funcName)</span><span class="s">s Time: </span><span class="si">%(asctime)</span><span class="s">s Message: </span><span class="si">%(message)</span><span class="s">s '''</span><span class="p">))</span> <span class="n">app</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">addHandler</span><span class="p">(</span><span class="n">mail_handler</span><span class="p">)</span> <span class="nd">@app.route</span><span class="p">(</span><span class="s">'/error'</span><span class="p">)</span> <span class="k">def</span> <span class="nf">error</span><span class="p">():</span> <span class="k">raise</span> <span class="nb">Exception</span><span class="p">(</span><span class="s">"Beds are burning!"</span><span class="p">)</span> <span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s">'__main__'</span><span class="p">:</span> <span class="c">#app.run(debug=True) #Handler isn't executed if app is run in debug mode</span> <span class="n">app</span><span class="o">.</span><span class="n">run</span><span class="p">()</span> </code></pre> </div> Loading translations from Google Spreadsheets 2015-09-04T11:16:30+02:00 https://fadeit.dk/blog /2015/09/04/managing-translations-with-gulp-or-grunt <h2 id="introduction">Introduction</h2> <p>This is a follow-up post to our <a href="https://fadeit.dk/blog/post/managing-angular-translate-translations">previous</a> translation management article. To recap - in that article we solved the issue of loading ‘fresh’ translation on development server using nginx and proxying translation requests to the api which in turn will fetch new translations from Google Spreadsheets. It can be seen as overkill - this flow complicates nginx configuration, slows down translation loading on every refresh and will make a ton of (unnecessary) requests while developing. While it’s nice to always get the latest translations, truth is that most of the time you don’t need to fetch translations from the cloud. In this post we’ll explore loading translations only during the build process, thus eliminating the need to proxy requests and write api endpoint in the first place.</p> <h2 id="workflow">Workflow</h2> <p>Since writing previous post, I have also changed my mind about lazy-loading translations using partial loader. This is primarily due to the lengths I had to go to cloak translations while they’re loading, especially considering that the size of the translations file is marginal for most applications. By packaging translations in the same minified file as the application I don’t have to worry about <a href="https://fadeit.dk/blog/post/preload-angular-translate-partial-loader">pre-loading</a> translations or flickering as they are being loaded. So we will introduce an extra task to our workflow that will fetch translations from Google Spreadsheets and converts it to a javascript file ready to be used by angular-translate.</p> <h2 id="using-gulp">Using Gulp</h2> <p>Although here at fadeit we started out with using Grunt exclusively, we’ve come to appreciate Gulp more in our recent projects. Primarily since writing custom tasks in good ol’ JavaScript is more intuitive as opposed to Grunt’s configuration approach. Surely piping the result of previous task directly to next one without having to write it to disk is a plus, but not in current context.</p> <div class="language-javascript highlighter-rouge"><pre class="highlight"><code><span class="kd">var</span> <span class="nx">gulp</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'gulp'</span><span class="p">);</span> <span class="kd">var</span> <span class="nx">request</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'request'</span><span class="p">);</span> <span class="kd">var</span> <span class="nx">fs</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'fs'</span><span class="p">);</span> <span class="nx">gulp</span><span class="p">.</span><span class="nx">task</span><span class="p">(</span><span class="s1">'translations'</span><span class="p">,</span> <span class="kd">function</span> <span class="p">(</span><span class="nx">cb</span><span class="p">)</span> <span class="p">{</span> <span class="kd">var</span> <span class="nx">uuid</span> <span class="o">=</span> <span class="s1">'1FsVuRLbtgxMZvWd4mpnKiAhqVYap-ZAx08LBeZ9HFJk'</span><span class="p">;</span> <span class="kd">var</span> <span class="nx">page</span> <span class="o">=</span> <span class="s1">'od6'</span> <span class="kd">var</span> <span class="nx">url</span> <span class="o">=</span> <span class="s1">'https://spreadsheets.google.com/feeds/list/'</span> <span class="o">+</span> <span class="nx">uuid</span> <span class="o">+</span> <span class="s1">'/'</span> <span class="o">+</span> <span class="nx">page</span> <span class="o">+</span> <span class="s1">'/public/values?alt=json'</span><span class="p">;</span> <span class="nx">request</span><span class="p">(</span><span class="nx">url</span><span class="p">,</span> <span class="kd">function</span><span class="p">(</span><span class="nx">err</span><span class="p">,</span> <span class="nx">res</span><span class="p">,</span> <span class="nx">body</span><span class="p">){</span> <span class="kd">var</span> <span class="nx">json</span> <span class="o">=</span> <span class="nx">JSON</span><span class="p">.</span><span class="nx">parse</span><span class="p">(</span><span class="nx">body</span><span class="p">);</span> <span class="kd">var</span> <span class="nx">translations</span> <span class="o">=</span> <span class="p">{</span> <span class="s1">'en'</span><span class="p">:</span> <span class="p">{},</span> <span class="s1">'da'</span><span class="p">:</span> <span class="p">{}</span> <span class="p">};</span> <span class="k">for</span><span class="p">(</span><span class="kd">var</span> <span class="nx">i</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="nx">i</span> <span class="o">&lt;</span> <span class="nx">json</span><span class="p">.</span><span class="nx">feed</span><span class="p">.</span><span class="nx">entry</span><span class="p">.</span><span class="nx">length</span><span class="p">;</span> <span class="nx">i</span><span class="o">++</span><span class="p">){</span> <span class="kd">var</span> <span class="nx">entry</span> <span class="o">=</span> <span class="nx">json</span><span class="p">.</span><span class="nx">feed</span><span class="p">.</span><span class="nx">entry</span><span class="p">[</span><span class="nx">i</span><span class="p">];</span> <span class="kd">var</span> <span class="nx">key</span> <span class="o">=</span> <span class="nx">entry</span><span class="p">.</span><span class="nx">gsx$key</span><span class="p">.</span><span class="nx">$t</span><span class="p">;</span> <span class="kd">var</span> <span class="nx">enValue</span> <span class="o">=</span> <span class="nx">entry</span><span class="p">.</span><span class="nx">gsx$en</span><span class="p">.</span><span class="nx">$t</span><span class="p">;</span> <span class="kd">var</span> <span class="nx">daValue</span> <span class="o">=</span> <span class="nx">entry</span><span class="p">.</span><span class="nx">gsx$da</span><span class="p">.</span><span class="nx">$t</span><span class="p">;</span> <span class="nx">translations</span><span class="p">.</span><span class="nx">en</span><span class="p">[</span><span class="nx">key</span><span class="p">]</span> <span class="o">=</span> <span class="nx">enValue</span><span class="p">;</span> <span class="nx">translations</span><span class="p">.</span><span class="nx">da</span><span class="p">[</span><span class="nx">key</span><span class="p">]</span> <span class="o">=</span> <span class="nx">daValue</span><span class="p">;</span> <span class="p">}</span> <span class="nx">writeTranslations</span><span class="p">(</span><span class="nx">translations</span><span class="p">,</span> <span class="nx">options</span><span class="p">.</span><span class="nx">src</span> <span class="o">+</span> <span class="s1">'/app/translations.js'</span><span class="p">,</span> <span class="nx">cb</span><span class="p">);</span> <span class="p">});</span> <span class="p">});</span> <span class="kd">function</span> <span class="nx">writeTranslations</span><span class="p">(</span><span class="nx">translations</span><span class="p">,</span> <span class="nx">file</span><span class="p">,</span> <span class="nx">cb</span><span class="p">){</span> <span class="kd">var</span> <span class="nx">contents</span> <span class="o">=</span> <span class="s2">"var appModule = angular.module('fadeit');\n"</span><span class="p">;</span> <span class="nx">contents</span> <span class="o">+=</span> <span class="s2">"appModule.config(function($translateProvider){\n"</span><span class="p">;</span> <span class="k">for</span><span class="p">(</span><span class="kd">var</span> <span class="nx">property</span> <span class="k">in</span> <span class="nx">translations</span><span class="p">){</span> <span class="kd">var</span> <span class="nx">lang</span> <span class="o">=</span> <span class="nx">translations</span><span class="p">[</span><span class="nx">property</span><span class="p">];</span> <span class="nx">contents</span> <span class="o">+=</span> <span class="s2">"$translateProvider.translations('"</span> <span class="o">+</span> <span class="nx">property</span> <span class="o">+</span> <span class="s2">"',\n"</span><span class="p">;</span> <span class="nx">contents</span> <span class="o">+=</span> <span class="nx">JSON</span><span class="p">.</span><span class="nx">stringify</span><span class="p">(</span><span class="nx">lang</span><span class="p">);</span> <span class="nx">contents</span> <span class="o">+=</span> <span class="s2">");\n"</span><span class="p">;</span> <span class="p">}</span> <span class="nx">contents</span> <span class="o">+=</span> <span class="s2">"});"</span><span class="p">;</span> <span class="nx">fs</span><span class="p">.</span><span class="nx">writeFile</span><span class="p">(</span><span class="nx">file</span><span class="p">,</span> <span class="nx">contents</span><span class="p">,</span> <span class="kd">function</span><span class="p">(</span><span class="nx">err</span><span class="p">){</span> <span class="k">if</span><span class="p">(</span><span class="nx">err</span><span class="p">){</span> <span class="nx">console</span><span class="p">.</span><span class="nx">error</span><span class="p">(</span><span class="s1">'Writing translations failed:'</span><span class="p">);</span> <span class="nx">console</span><span class="p">.</span><span class="nx">error</span><span class="p">(</span><span class="nx">err</span><span class="p">);</span> <span class="p">}</span> <span class="nx">cb</span><span class="p">();</span> <span class="p">});</span> <span class="p">}</span> </code></pre> </div> <p>As seen from the code, couple modules are used - namely <code class="highlighter-rouge">request</code> to fetch the JSON data and <code class="highlighter-rouge">fs</code> to write a file. The task won’t only write the file, but it will also generate an angular.js module file that will configure translations ready to be used. It can be seen a bit hacky to generate javascript file like that, but I’ve yet to come up with more clean solution. The upside is that this code is not prone to change and I haven’t touched it since writing it. There’s caveat though - if you are using <a href="http://www.browsersync.io/">browsersync</a>, then it may trigger refresh before the translations are actually pulled (because other files changed before). To fix this, we need to make sure that translations are loaded before any other tasks take place. It is easily solved with gulp-sync module, to execute certain tasks synchronously. Same would also apply if the process will concatenate files together - we need to make sure new translations are loaded before that happens. In the sample build process below I’ve split up the build process in two portions where each of them is executed asynchronously. The callback at the end of the gulp task is necessary so that gulp-sync knows when to start processing next batch of tasks.</p> <div class="language-javascript highlighter-rouge"><pre class="highlight"><code><span class="kd">var</span> <span class="nx">gulpsync</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'gulp-sync'</span><span class="p">)(</span><span class="nx">gulp</span><span class="p">);</span> <span class="nx">gulp</span><span class="p">.</span><span class="nx">task</span><span class="p">(</span><span class="s1">'build'</span><span class="p">,</span> <span class="nx">gulpsync</span><span class="p">.</span><span class="nx">sync</span><span class="p">([[</span><span class="s1">'translations'</span><span class="p">,</span> <span class="s1">'clean'</span><span class="p">],</span> <span class="p">[</span><span class="s1">'html'</span><span class="p">,</span> <span class="s1">'scripts'</span><span class="p">]]));</span> </code></pre> </div> <h2 id="using-grunt">Using Grunt</h2> <p>Firstly I must apologise as I did not take time to develop a pure grunt way of doing so. This is because it started out as a python script which I later incorporated in the grunt workflow. The script will serve same purpose as the one above, with a minor difference of calling an external python executeable instead.</p> <div class="language-python highlighter-rouge"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">json</span> <span class="kn">import</span> <span class="nn">urllib.request</span> <span class="kn">from</span> <span class="nn">collections</span> <span class="kn">import</span> <span class="n">OrderedDict</span> <span class="n">sheet_id</span> <span class="o">=</span> <span class="s">'1FsVuRLbtgxMZvWd4mpnKiAhqVYap-ZAx08LBeZ9HFJk'</span><span class="p">;</span> <span class="n">page_id</span> <span class="o">=</span> <span class="s">'od6'</span> <span class="n">base</span> <span class="o">=</span> <span class="s">'https://spreadsheets.google.com/feeds/list/{0}/{1}/public/values?alt=json'</span> <span class="n">translation_config</span> <span class="o">=</span> <span class="s">'src/app/translations.js'</span> <span class="n">url</span> <span class="o">=</span> <span class="n">base</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">sheet_id</span><span class="p">,</span> <span class="n">page_id</span><span class="p">)</span> <span class="n">json_data</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">loads</span><span class="p">(</span><span class="n">urllib</span><span class="o">.</span><span class="n">request</span><span class="o">.</span><span class="n">urlopen</span><span class="p">(</span><span class="n">url</span><span class="p">)</span><span class="o">.</span><span class="n">read</span><span class="p">()</span><span class="o">.</span><span class="n">decode</span><span class="p">(</span><span class="s">'utf8'</span><span class="p">))</span> <span class="c">#we need to build different data structure</span> <span class="n">en_translation_map</span> <span class="o">=</span> <span class="n">OrderedDict</span><span class="p">()</span> <span class="n">da_translation_map</span> <span class="o">=</span> <span class="n">OrderedDict</span><span class="p">()</span> <span class="k">for</span> <span class="n">entry</span> <span class="ow">in</span> <span class="n">json_data</span><span class="p">[</span><span class="s">'feed'</span><span class="p">][</span><span class="s">'entry'</span><span class="p">]:</span> <span class="n">key</span> <span class="o">=</span> <span class="n">entry</span><span class="p">[</span><span class="s">'gsx$key'</span><span class="p">][</span><span class="s">'$t'</span><span class="p">]</span> <span class="n">en_value</span> <span class="o">=</span> <span class="n">entry</span><span class="p">[</span><span class="s">'gsx$en'</span><span class="p">][</span><span class="s">'$t'</span><span class="p">]</span> <span class="n">da_value</span> <span class="o">=</span> <span class="n">entry</span><span class="p">[</span><span class="s">'gsx$da'</span><span class="p">][</span><span class="s">'$t'</span><span class="p">]</span> <span class="n">en_translation_map</span><span class="p">[</span><span class="n">key</span><span class="p">]</span> <span class="o">=</span> <span class="n">en_value</span> <span class="n">da_translation_map</span><span class="p">[</span><span class="n">key</span><span class="p">]</span> <span class="o">=</span> <span class="n">da_value</span> <span class="c">#Javascript for translations file we are generating</span> <span class="n">config</span> <span class="o">=</span> <span class="s">""" var appModule = angular.module('fadeit'); appModule.config(function($translateProvider){ """</span> <span class="n">en_translations</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">en_translation_map</span><span class="p">,</span> <span class="n">indent</span><span class="o">=</span><span class="mi">4</span><span class="p">)</span> <span class="n">da_translations</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">da_translation_map</span><span class="p">,</span> <span class="n">indent</span><span class="o">=</span><span class="mi">4</span><span class="p">)</span> <span class="n">config</span> <span class="o">=</span> <span class="n">config</span> <span class="o">+</span> <span class="s">"</span><span class="se">\n</span><span class="s">$translateProvider.translations('en',"</span> <span class="o">+</span> <span class="n">en_translations</span> <span class="o">+</span> <span class="s">');'</span> <span class="n">config</span> <span class="o">=</span> <span class="n">config</span> <span class="o">+</span> <span class="s">"</span><span class="se">\n</span><span class="s">$translateProvider.translations('da',"</span> <span class="o">+</span> <span class="n">da_translations</span> <span class="o">+</span> <span class="s">');'</span> <span class="n">config</span> <span class="o">=</span> <span class="n">config</span> <span class="o">+</span> <span class="s">"</span><span class="se">\n</span><span class="s">});"</span> <span class="c">#close function</span> <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">translation_config</span><span class="p">,</span> <span class="s">'w'</span><span class="p">)</span> <span class="k">as</span> <span class="n">outfile</span><span class="p">:</span> <span class="n">outfile</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">config</span><span class="p">)</span> </code></pre> </div> <p>Now we can save this file in the project (in our case src/translations.py) and call it using grunt-shell during build time.</p> <div class="language-javascript highlighter-rouge"><pre class="highlight"><code><span class="nx">shell</span><span class="err">:</span> <span class="p">{</span> <span class="nl">translations</span><span class="p">:</span> <span class="p">{</span> <span class="nl">options</span><span class="p">:</span> <span class="p">{</span> <span class="nl">stdout</span><span class="p">:</span> <span class="kc">true</span> <span class="p">},</span> <span class="nx">command</span><span class="err">:</span> <span class="s1">'python3 src/translations.py'</span> <span class="p">}</span> <span class="p">}</span> </code></pre> </div> <p>Since grunt executes tasks synchronously, all there is left to do is call this task in the beginning of the build process and it’ll generate a following file.</p> <div class="language-javascript highlighter-rouge"><pre class="highlight"><code><span class="kd">var</span> <span class="nx">appModule</span> <span class="o">=</span> <span class="nx">angular</span><span class="p">.</span><span class="nx">module</span><span class="p">(</span><span class="s1">'fadeit'</span><span class="p">);</span> <span class="nx">appModule</span><span class="p">.</span><span class="nx">config</span><span class="p">(</span><span class="kd">function</span><span class="p">(</span><span class="nx">$translateProvider</span><span class="p">)</span> <span class="p">{</span> <span class="nx">$translateProvider</span><span class="p">.</span><span class="nx">translations</span><span class="p">(</span><span class="s1">'en'</span><span class="p">,</span> <span class="p">{</span> <span class="s2">"HELLO"</span><span class="p">:</span> <span class="s2">"Hello"</span> <span class="p">}</span> <span class="p">);</span> <span class="nx">$translateProvider</span><span class="p">.</span><span class="nx">translations</span><span class="p">(</span><span class="s1">'da'</span><span class="p">,</span> <span class="p">{</span> <span class="s2">"HELLO"</span><span class="p">:</span> <span class="s2">"Hej"</span> <span class="p">}</span> <span class="p">);</span> <span class="p">});</span> </code></pre> </div> <h2 id="closing-words">Closing words</h2> <p>Over-all I’ve come to prefer loading translations during build time instead. While it will add few seconds to the build process, I will make it up by not having to wait for translations being proxied on every reload. If I already have a server running and just wish to reload translations, I can issue the command directly instead of re-building everything. For gulp it would be <code class="highlighter-rouge">gulp translations</code> and for grunt <code class="highlighter-rouge">grunt shell:translations</code>. I’ve also decided to ditch versioning translations and generally will add <code class="highlighter-rouge">src/translations.js</code> to <code class="highlighter-rouge">.gitignore</code>.</p> Serving generated images from memory 2015-04-30T14:15:00+02:00 https://fadeit.dk/blog /2015/04/30/python3-flask-pil-in-memory-image <p>In this post, we are going to implement a simple image placeholder service like <a href="https://placehold.it/">placehold.it</a>. Use-cases for disposable images extend beyond placeholder services, commonly used to encode text into an image in order to protect websites from being scraped for valuable information.</p> <p>A simple approach could be to generate an image, write it to disc and then serve the static file. One could improve the solution by using temporary files, however the process still remains inefficient. The solution, of course, is to use in-memory files to cut out the process of writing the file to the hard drive.</p> <p>We are using Pillow, a fork of Python’s PIL (Python Imaging Library) that is actively maintained. While there is information out there on how to write an image into memory using StringIO module for Python2, using Python3 version io.StringIO results in an error:</p> <div class="language-python highlighter-rouge"><pre class="highlight"><code><span class="n">Traceback</span> <span class="p">(</span><span class="n">most</span> <span class="n">recent</span> <span class="n">call</span> <span class="n">last</span><span class="p">):</span> <span class="n">File</span> <span class="s">"image.py"</span><span class="p">,</span> <span class="n">line</span> <span class="mi">10</span><span class="p">,</span> <span class="ow">in</span> <span class="o">&lt;</span><span class="n">module</span><span class="o">&gt;</span> <span class="n">image</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="n">string_io</span><span class="p">,</span> <span class="s">'PNG'</span><span class="p">)</span> <span class="n">File</span> <span class="s">"~/.pyenv/versions/3.4.0/lib/python3.4/site-packages/PIL/Image.py"</span><span class="p">,</span> <span class="n">line</span> <span class="mi">1685</span><span class="p">,</span> <span class="ow">in</span> <span class="n">save</span> <span class="n">save_handler</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">fp</span><span class="p">,</span> <span class="n">filename</span><span class="p">)</span> <span class="n">File</span> <span class="s">"~/.pyenv/versions/3.4.0/lib/python3.4/site-packages/PIL/PngImagePlugin.py"</span><span class="p">,</span> <span class="n">line</span> <span class="mi">631</span><span class="p">,</span> <span class="ow">in</span> <span class="n">_save</span> <span class="n">fp</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">_MAGIC</span><span class="p">)</span> <span class="nb">TypeError</span><span class="p">:</span> <span class="n">string</span> <span class="n">argument</span> <span class="n">expected</span><span class="p">,</span> <span class="n">got</span> <span class="s">'bytes'</span> </code></pre> </div> <p>As hinted in the error message, the solution is to use io.BytesIO instead. See a minimal working sample below:</p> <div class="language-python highlighter-rouge"><pre class="highlight"><code><span class="kn">from</span> <span class="nn">io</span> <span class="kn">import</span> <span class="n">BytesIO</span> <span class="kn">from</span> <span class="nn">PIL</span> <span class="kn">import</span> <span class="n">Image</span><span class="p">,</span> <span class="n">ImageDraw</span> <span class="n">image</span> <span class="o">=</span> <span class="n">Image</span><span class="o">.</span><span class="n">new</span><span class="p">(</span><span class="s">"RGB"</span><span class="p">,</span> <span class="p">(</span><span class="mi">300</span><span class="p">,</span> <span class="mi">50</span><span class="p">))</span> <span class="n">draw</span> <span class="o">=</span> <span class="n">ImageDraw</span><span class="o">.</span><span class="n">Draw</span><span class="p">(</span><span class="n">image</span><span class="p">)</span> <span class="n">draw</span><span class="o">.</span><span class="n">text</span><span class="p">((</span><span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">),</span> <span class="s">"This text is drawn on image"</span><span class="p">)</span> <span class="n">byte_io</span> <span class="o">=</span> <span class="n">BytesIO</span><span class="p">()</span> <span class="n">image</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="n">byte_io</span><span class="p">,</span> <span class="s">'PNG'</span><span class="p">)</span> </code></pre> </div> <p>Putting it all together, let’s implement a simple Flask service that takes one parameter, extracts dimensions, generates an image and writes the passed parameter as a text on the image. Flask is amazing, allowing us to write fully functional web service in just 30 lines of code:</p> <div class="language-python highlighter-rouge"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">re</span> <span class="kn">from</span> <span class="nn">io</span> <span class="kn">import</span> <span class="n">BytesIO</span> <span class="kn">from</span> <span class="nn">flask</span> <span class="kn">import</span> <span class="n">Flask</span><span class="p">,</span> <span class="n">abort</span><span class="p">,</span> <span class="n">send_file</span> <span class="kn">from</span> <span class="nn">PIL</span> <span class="kn">import</span> <span class="n">Image</span><span class="p">,</span> <span class="n">ImageDraw</span> <span class="n">app</span> <span class="o">=</span> <span class="n">Flask</span><span class="p">(</span><span class="n">__name__</span><span class="p">)</span> <span class="nd">@app.route</span><span class="p">(</span><span class="s">'/&lt;dimensions&gt;'</span><span class="p">)</span> <span class="k">def</span> <span class="nf">generate_image</span><span class="p">(</span><span class="n">dimensions</span><span class="p">):</span> <span class="c">#Extract digits from request variable e.g 200x300</span> <span class="n">sizes</span> <span class="o">=</span> <span class="p">[</span><span class="nb">int</span><span class="p">(</span><span class="n">s</span><span class="p">)</span> <span class="k">for</span> <span class="n">s</span> <span class="ow">in</span> <span class="n">re</span><span class="o">.</span><span class="n">findall</span><span class="p">(</span><span class="s">r'</span><span class="err">\</span><span class="s">d+'</span><span class="p">,</span> <span class="n">dimensions</span><span class="p">)]</span> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">sizes</span><span class="p">)</span> <span class="o">!=</span> <span class="mi">2</span><span class="p">:</span> <span class="n">abort</span><span class="p">(</span><span class="mi">400</span><span class="p">)</span> <span class="n">width</span> <span class="o">=</span> <span class="n">sizes</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="n">height</span> <span class="o">=</span> <span class="n">sizes</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="n">image</span> <span class="o">=</span> <span class="n">Image</span><span class="o">.</span><span class="n">new</span><span class="p">(</span><span class="s">"RGB"</span><span class="p">,</span> <span class="p">(</span><span class="n">width</span><span class="p">,</span> <span class="n">height</span><span class="p">))</span> <span class="n">draw</span> <span class="o">=</span> <span class="n">ImageDraw</span><span class="o">.</span><span class="n">Draw</span><span class="p">(</span><span class="n">image</span><span class="p">)</span> <span class="c">#Position text roughly in the center</span> <span class="n">draw</span><span class="o">.</span><span class="n">text</span><span class="p">((</span><span class="n">width</span><span class="o">/</span><span class="mi">2</span> <span class="o">-</span> <span class="mi">25</span><span class="p">,</span> <span class="n">height</span><span class="o">/</span><span class="mi">2</span> <span class="o">-</span> <span class="mi">5</span><span class="p">),</span> <span class="n">dimensions</span><span class="p">)</span> <span class="n">byte_io</span> <span class="o">=</span> <span class="n">BytesIO</span><span class="p">()</span> <span class="n">image</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="n">byte_io</span><span class="p">,</span> <span class="s">'PNG'</span><span class="p">)</span> <span class="n">byte_io</span><span class="o">.</span><span class="n">seek</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span> <span class="k">return</span> <span class="n">send_file</span><span class="p">(</span><span class="n">byte_io</span><span class="p">,</span> <span class="n">mimetype</span><span class="o">=</span><span class="s">'image/png'</span><span class="p">)</span> <span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s">'__main__'</span><span class="p">:</span> <span class="n">app</span><span class="o">.</span><span class="n">run</span><span class="p">(</span><span class="n">debug</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span> </code></pre> </div> <p>To run this code, you may need to install few dependencies:</p> <div class="language-bash highlighter-rouge"><pre class="highlight"><code><span class="c">#Install dependencies</span> sudo apt-get install virtualenv python3.4-dev python-pip <span class="c">#Create virtual environment</span> virtualenv -p /usr/bin/python3.4 venv <span class="c">#Activate virtual environment</span> <span class="nb">source </span>venv/bin/activate <span class="c">#Install libraries</span> pip install flask Pillow <span class="c">#Run server</span> python server.py <span class="c">#http://localhost:5000/200x300</span> </code></pre> </div> <p><img src="/blog/assets/python3-flask-pil-in-memory-image/sample.png" alt="Generated image" /></p>