- flask 2017-03-19T16:58:15+01:00 https://fadeit.dk/blog/tag/flask.html Python logging handler for Mailgun 2015-10-12T13:25:30+02:00 https://fadeit.dk/blog /2015/10/12/mailgun-python-log-handler <p>Yet another reason to love python is the wonderful logging API provided by a standard library. Having a decent logger provided by the language saves developers time from integrating with some third-party library, but most importantly it contributes to a clean system where all modules use the same logger. Out of the box, there is a <a href="https://docs.python.org/3/library/logging.handlers.html">handler</a> for almost any typical use-case, but the best feature is the option to create your own with relative ease. So-far we’ve been using <a href="https://docs.python.org/3/library/logging.handlers.html#smtphandler">SMTPHandler</a>, but since moving to <a href="https://www.mailgun.com/">Mailgun</a>, I decided to experiment with a custom handler to submit logs via their API instead. I’d like to mention that creating a custom logging handler is not a requirement, it is still possible to use their SMTP server with SMTP handler. To get started, we need to extend Handler baseclass and override emit method to make a POST request.</p> <div class="language-python highlighter-rouge"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">logging</span> <span class="kn">import</span> <span class="nn">requests</span> <span class="k">class</span> <span class="nc">MailgunHandler</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">Handler</span><span class="p">):</span> <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">api_url</span><span class="p">,</span> <span class="n">api_key</span><span class="p">,</span> <span class="n">sender</span><span class="p">,</span> <span class="n">recipients</span><span class="p">,</span> <span class="n">subject</span><span class="p">):</span> <span class="c"># run the regular Handler __init__</span> <span class="n">logging</span><span class="o">.</span><span class="n">Handler</span><span class="o">.</span><span class="n">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="bp">self</span><span class="o">.</span><span class="n">api_url</span> <span class="o">=</span> <span class="n">api_url</span> <span class="bp">self</span><span class="o">.</span><span class="n">api_key</span> <span class="o">=</span> <span class="n">api_key</span> <span class="bp">self</span><span class="o">.</span><span class="n">sender</span> <span class="o">=</span> <span class="n">sender</span> <span class="bp">self</span><span class="o">.</span><span class="n">recipients</span> <span class="o">=</span> <span class="n">recipients</span> <span class="bp">self</span><span class="o">.</span><span class="n">subject</span> <span class="o">=</span> <span class="n">subject</span> <span class="k">def</span> <span class="nf">emit</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">record</span><span class="p">):</span> <span class="c"># record.message is the log message</span> <span class="k">for</span> <span class="n">recipient</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">recipients</span><span class="p">:</span> <span class="n">data</span> <span class="o">=</span> <span class="p">{</span> <span class="s">"from"</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">sender</span><span class="p">,</span> <span class="s">"to"</span><span class="p">:</span> <span class="n">recipient</span><span class="p">,</span> <span class="s">"subject"</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">subject</span><span class="p">,</span> <span class="s">"text"</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">record</span><span class="p">)</span> <span class="p">}</span> <span class="n">requests</span><span class="o">.</span><span class="n">post</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">api_url</span><span class="p">,</span> <span class="n">auth</span><span class="o">=</span><span class="p">(</span><span class="s">"api"</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">api_key</span><span class="p">),</span> <span class="n">data</span><span class="o">=</span><span class="n">data</span><span class="p">)</span> </code></pre> </div> <p>I have created a simple flask application that registers the handler and also implemented a route that triggers an error to test it out.</p> <div class="language-python highlighter-rouge"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">logging</span> <span class="kn">from</span> <span class="nn">flask</span> <span class="kn">import</span> <span class="n">Flask</span> <span class="kn">from</span> <span class="nn">mailgun_handler</span> <span class="kn">import</span> <span class="n">MailgunHandler</span> <span class="n">app</span> <span class="o">=</span> <span class="n">Flask</span><span class="p">(</span><span class="n">__name__</span><span class="p">)</span> <span class="c"># Setup logging handler</span> <span class="n">mail_handler</span> <span class="o">=</span> <span class="n">MailgunHandler</span><span class="p">(</span> <span class="n">api_url</span><span class="o">=</span><span class="s">'https://api.mailgun.net/v3/fadeit.dk/messages'</span><span class="p">,</span> <span class="n">api_key</span><span class="o">=</span><span class="s">'secret'</span><span class="p">,</span> <span class="n">sender</span><span class="o">=</span><span class="s">'noreply@fadeit.dk'</span><span class="p">,</span> <span class="n">recipients</span><span class="o">=</span><span class="p">[</span><span class="s">'ss@fadeit.dk'</span><span class="p">,</span> <span class="s">'ja@fadeit.dk'</span><span class="p">],</span> <span class="n">subject</span><span class="o">=</span><span class="s">'Application error!'</span> <span class="p">)</span> <span class="n">mail_handler</span><span class="o">.</span><span class="n">setLevel</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">ERROR</span><span class="p">)</span> <span class="n">mail_handler</span><span class="o">.</span><span class="n">setFormatter</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">Formatter</span><span class="p">(</span><span class="s">''' Message type: </span><span class="si">%(levelname)</span><span class="s">s Location: </span><span class="si">%(pathname)</span><span class="s">s:</span><span class="si">%(lineno)</span><span class="s">d Module: </span><span class="si">%(module)</span><span class="s">s Function: </span><span class="si">%(funcName)</span><span class="s">s Time: </span><span class="si">%(asctime)</span><span class="s">s Message: </span><span class="si">%(message)</span><span class="s">s '''</span><span class="p">))</span> <span class="n">app</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">addHandler</span><span class="p">(</span><span class="n">mail_handler</span><span class="p">)</span> <span class="nd">@app.route</span><span class="p">(</span><span class="s">'/error'</span><span class="p">)</span> <span class="k">def</span> <span class="nf">error</span><span class="p">():</span> <span class="k">raise</span> <span class="nb">Exception</span><span class="p">(</span><span class="s">"Beds are burning!"</span><span class="p">)</span> <span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s">'__main__'</span><span class="p">:</span> <span class="c">#app.run(debug=True) #Handler isn't executed if app is run in debug mode</span> <span class="n">app</span><span class="o">.</span><span class="n">run</span><span class="p">()</span> </code></pre> </div> Serving generated images from memory 2015-04-30T14:15:00+02:00 https://fadeit.dk/blog /2015/04/30/python3-flask-pil-in-memory-image <p>In this post, we are going to implement a simple image placeholder service like <a href="https://placehold.it/">placehold.it</a>. Use-cases for disposable images extend beyond placeholder services, commonly used to encode text into an image in order to protect websites from being scraped for valuable information.</p> <p>A simple approach could be to generate an image, write it to disc and then serve the static file. One could improve the solution by using temporary files, however the process still remains inefficient. The solution, of course, is to use in-memory files to cut out the process of writing the file to the hard drive.</p> <p>We are using Pillow, a fork of Python’s PIL (Python Imaging Library) that is actively maintained. While there is information out there on how to write an image into memory using StringIO module for Python2, using Python3 version io.StringIO results in an error:</p> <div class="language-python highlighter-rouge"><pre class="highlight"><code><span class="n">Traceback</span> <span class="p">(</span><span class="n">most</span> <span class="n">recent</span> <span class="n">call</span> <span class="n">last</span><span class="p">):</span> <span class="n">File</span> <span class="s">"image.py"</span><span class="p">,</span> <span class="n">line</span> <span class="mi">10</span><span class="p">,</span> <span class="ow">in</span> <span class="o">&lt;</span><span class="n">module</span><span class="o">&gt;</span> <span class="n">image</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="n">string_io</span><span class="p">,</span> <span class="s">'PNG'</span><span class="p">)</span> <span class="n">File</span> <span class="s">"~/.pyenv/versions/3.4.0/lib/python3.4/site-packages/PIL/Image.py"</span><span class="p">,</span> <span class="n">line</span> <span class="mi">1685</span><span class="p">,</span> <span class="ow">in</span> <span class="n">save</span> <span class="n">save_handler</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">fp</span><span class="p">,</span> <span class="n">filename</span><span class="p">)</span> <span class="n">File</span> <span class="s">"~/.pyenv/versions/3.4.0/lib/python3.4/site-packages/PIL/PngImagePlugin.py"</span><span class="p">,</span> <span class="n">line</span> <span class="mi">631</span><span class="p">,</span> <span class="ow">in</span> <span class="n">_save</span> <span class="n">fp</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">_MAGIC</span><span class="p">)</span> <span class="nb">TypeError</span><span class="p">:</span> <span class="n">string</span> <span class="n">argument</span> <span class="n">expected</span><span class="p">,</span> <span class="n">got</span> <span class="s">'bytes'</span> </code></pre> </div> <p>As hinted in the error message, the solution is to use io.BytesIO instead. See a minimal working sample below:</p> <div class="language-python highlighter-rouge"><pre class="highlight"><code><span class="kn">from</span> <span class="nn">io</span> <span class="kn">import</span> <span class="n">BytesIO</span> <span class="kn">from</span> <span class="nn">PIL</span> <span class="kn">import</span> <span class="n">Image</span><span class="p">,</span> <span class="n">ImageDraw</span> <span class="n">image</span> <span class="o">=</span> <span class="n">Image</span><span class="o">.</span><span class="n">new</span><span class="p">(</span><span class="s">"RGB"</span><span class="p">,</span> <span class="p">(</span><span class="mi">300</span><span class="p">,</span> <span class="mi">50</span><span class="p">))</span> <span class="n">draw</span> <span class="o">=</span> <span class="n">ImageDraw</span><span class="o">.</span><span class="n">Draw</span><span class="p">(</span><span class="n">image</span><span class="p">)</span> <span class="n">draw</span><span class="o">.</span><span class="n">text</span><span class="p">((</span><span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">),</span> <span class="s">"This text is drawn on image"</span><span class="p">)</span> <span class="n">byte_io</span> <span class="o">=</span> <span class="n">BytesIO</span><span class="p">()</span> <span class="n">image</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="n">byte_io</span><span class="p">,</span> <span class="s">'PNG'</span><span class="p">)</span> </code></pre> </div> <p>Putting it all together, let’s implement a simple Flask service that takes one parameter, extracts dimensions, generates an image and writes the passed parameter as a text on the image. Flask is amazing, allowing us to write fully functional web service in just 30 lines of code:</p> <div class="language-python highlighter-rouge"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">re</span> <span class="kn">from</span> <span class="nn">io</span> <span class="kn">import</span> <span class="n">BytesIO</span> <span class="kn">from</span> <span class="nn">flask</span> <span class="kn">import</span> <span class="n">Flask</span><span class="p">,</span> <span class="n">abort</span><span class="p">,</span> <span class="n">send_file</span> <span class="kn">from</span> <span class="nn">PIL</span> <span class="kn">import</span> <span class="n">Image</span><span class="p">,</span> <span class="n">ImageDraw</span> <span class="n">app</span> <span class="o">=</span> <span class="n">Flask</span><span class="p">(</span><span class="n">__name__</span><span class="p">)</span> <span class="nd">@app.route</span><span class="p">(</span><span class="s">'/&lt;dimensions&gt;'</span><span class="p">)</span> <span class="k">def</span> <span class="nf">generate_image</span><span class="p">(</span><span class="n">dimensions</span><span class="p">):</span> <span class="c">#Extract digits from request variable e.g 200x300</span> <span class="n">sizes</span> <span class="o">=</span> <span class="p">[</span><span class="nb">int</span><span class="p">(</span><span class="n">s</span><span class="p">)</span> <span class="k">for</span> <span class="n">s</span> <span class="ow">in</span> <span class="n">re</span><span class="o">.</span><span class="n">findall</span><span class="p">(</span><span class="s">r'</span><span class="err">\</span><span class="s">d+'</span><span class="p">,</span> <span class="n">dimensions</span><span class="p">)]</span> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">sizes</span><span class="p">)</span> <span class="o">!=</span> <span class="mi">2</span><span class="p">:</span> <span class="n">abort</span><span class="p">(</span><span class="mi">400</span><span class="p">)</span> <span class="n">width</span> <span class="o">=</span> <span class="n">sizes</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="n">height</span> <span class="o">=</span> <span class="n">sizes</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="n">image</span> <span class="o">=</span> <span class="n">Image</span><span class="o">.</span><span class="n">new</span><span class="p">(</span><span class="s">"RGB"</span><span class="p">,</span> <span class="p">(</span><span class="n">width</span><span class="p">,</span> <span class="n">height</span><span class="p">))</span> <span class="n">draw</span> <span class="o">=</span> <span class="n">ImageDraw</span><span class="o">.</span><span class="n">Draw</span><span class="p">(</span><span class="n">image</span><span class="p">)</span> <span class="c">#Position text roughly in the center</span> <span class="n">draw</span><span class="o">.</span><span class="n">text</span><span class="p">((</span><span class="n">width</span><span class="o">/</span><span class="mi">2</span> <span class="o">-</span> <span class="mi">25</span><span class="p">,</span> <span class="n">height</span><span class="o">/</span><span class="mi">2</span> <span class="o">-</span> <span class="mi">5</span><span class="p">),</span> <span class="n">dimensions</span><span class="p">)</span> <span class="n">byte_io</span> <span class="o">=</span> <span class="n">BytesIO</span><span class="p">()</span> <span class="n">image</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="n">byte_io</span><span class="p">,</span> <span class="s">'PNG'</span><span class="p">)</span> <span class="n">byte_io</span><span class="o">.</span><span class="n">seek</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span> <span class="k">return</span> <span class="n">send_file</span><span class="p">(</span><span class="n">byte_io</span><span class="p">,</span> <span class="n">mimetype</span><span class="o">=</span><span class="s">'image/png'</span><span class="p">)</span> <span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s">'__main__'</span><span class="p">:</span> <span class="n">app</span><span class="o">.</span><span class="n">run</span><span class="p">(</span><span class="n">debug</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span> </code></pre> </div> <p>To run this code, you may need to install few dependencies:</p> <div class="language-bash highlighter-rouge"><pre class="highlight"><code><span class="c">#Install dependencies</span> sudo apt-get install virtualenv python3.4-dev python-pip <span class="c">#Create virtual environment</span> virtualenv -p /usr/bin/python3.4 venv <span class="c">#Activate virtual environment</span> <span class="nb">source </span>venv/bin/activate <span class="c">#Install libraries</span> pip install flask Pillow <span class="c">#Run server</span> python server.py <span class="c">#http://localhost:5000/200x300</span> </code></pre> </div> <p><img src="/blog/assets/python3-flask-pil-in-memory-image/sample.png" alt="Generated image" /></p>