Tagged: programming

These blog posts have all been tagged with programming. These tags appear to be related:

31 Oct
2016
Blender Conference 2016

BConf 2016 was awesome, as ever. This is just a blog post to share the slides of my workshop "Develop add-ons like a pro".


17 Mar
2016
Python-RSA 3.4 released

Python-RSA 3.4 has been released! This is the first release from GitHub, and contains a lot of improvements by myself, Michael Manganiello, and Filippo Valsorda, including the solution for a side-channel vulnerability.

Get it while it's hot at the Python Package Index, or just install with pip install --upgrade rsa.

Click here for more details and a plan for the future.


04 Feb
2016
My name in the modern world

Frustration

My last name is a technical hurdle. At least, for many people. In this high-tech, modern, globalized society, one would expect that a simple accented character shouldn't be an issue. After all, even a small child can write Stüvel correctly with a stick in the mud. So why is it such an issue in the world of computers?


18 Jan
2016
New website & Python-RSA 3.3

You won't notice it, but you're now on my new website. The look is still the same, but under the hood a lot has changed. This includes the blog being fixed again, so that I can actually write new posts! Can you imagine, I didn't blog a single thing in 2015.... well, that'll change.

Python-RSA 3.3 has been released! This release fixes a security leak, which was filed under CVE-2016-1494. As always, you can get the new version from the Python Package Index.


06 Nov
2014
Python FlickrAPI 2.0 released

Python FlickrAPI library version 2.0 has been released, and can be downloaded from the Python Package Index with more info at my FlickrAPI page.

The major change is that the "new" OAuth-based authentication is used. This makes our Python FlickrAPI library a lot more future-proof.

Thanks to Jim Easterbrook, Thai Nguyen, Nick Loadholtes and Bengt for their contributions.


30 Jul
2014
Moving from pwsafe to KeePass

My favourite password management tool, pwsafe, is getting old. It's no longer part of Ubuntu and I've had to compile it myself for quite a while now. It's also not supported on Android, and shows other signs of aging. So, I moved to KeePass. Exporting my database to CSV was simple, and to make the job of importing it correctly into KeePass even easier, I wrote a simple Python script.

Click to see the code...


10 Mar
2014
Reloading your code in Blender

Blender lets you script more or less everything. You can have your code inside Blender (and the .blend file) itself, but I prefer to store it outside the .blend file. This makes it easier to handle changes, and allows you to use any editor you want. The downside is that reloading your code becomes a bit of a hassle. This is a little snippet I wrote to reload my code in Blender with the click of a button.

Click to see the code...


28 Feb
2014
Counting pixels in Blender

For my research, I'm using Blender to generate images. I wanted to know how visible a certain object is in the final render (i.e. how many pixels it occupies). For this there is the "object index" render pass (aka "IndexOB" in the compositor). I've been struggling with it, since it always outputs that the index is 0, even though there are multiple objects in the scene.

Well, with the help of mfoxdogg on the #blender IRC channel, we found a solution: You need to set the index by hand, for every object you're interested in. If you go to the object properties (in the properties explorer), in the section "Relations" there is a slider "Pass Index". This is set to 0 by default, and you can set it to any positive number you want. This is then reflected in the output of the "IndexOB" render pass.

Click to see how I counted the pixels...


22 Feb
2014
Python-RSA version 3.1.4 released

Hacking, the analog way

It's been a while since I've been working on my RSA library, and in the last weeks I've released a few new minor versions. Here are the differences since my last blog post about it:

  • Fixed Python 3.x incompatibilities.

  • Dropped support for Python 2.5.

  • Added support for loading public keys from OpenSSL

  • rsa.pkcs1.verify() returns True when successful (thanks Tim Heckman)

  • First updated Distribute (thanks Jason R. Coombs), later replaced Distribute with Setuptools.

  • Fixed typo in pyrsa-verify help message


30 Oct
2013
Blender conference 2013

Last weekend Amsterdam was visited by a great group of over 250 Blender professionals and enthousiast, for the Blender Conference 2013. This year it was my pleasure to give a workshop about my research, and how I use Blender to visualize my result.

For the attendants (and other interested people), I've put my presentation, the Blender file and accompanying SQLite database online: https://stuvel.eu/files/bcon_2013_sybren_workshop.7z

You can still see the workshop online on Youtube: https://www.youtube.com/watch?v=Wt8WdSvaRms&feature=player_detailpage#t=620 (the first 10 minutes is just people leaving the previous workshop, and me setting up).

click for the video...


09 Dec
2012
Start XBMC from remote

The Android XBMC Remote app has a "Power on" button. This button sends a wake-on-lan (WOL) packet to the XBMC server, so that it can wake up. However, my XBMC server runs all kinds of server stuff, so I don't want to let it sleep. Still, XBMC uses more than 10% CPU power even when it's in idle, so I don't want it running all the time either.

I've written a small Python script that executes a command every time it receives a WOL packet. This is then used to start XBMC.

Click for the source code...


17 Jun
2012
Python-RSA version 3.1.1 released

Hacking, the analog way

Python-RSA version 3.1.1 has been released. A lot of things have been improved, thanks to Yesudeep Mangalapilly.

  • Much, much faster

  • Compatibility with Python 3.2

  • Lot of bug fixes

Check the Python-RSA page for more information!

Observant readers may have noticed that I snuck in a tiny minor release. Version 3.1.1 introduces full Python 3.2 compatibility for the key saving/loading stuff as well, and fixes some doctests for Python 2.7. Nothing in the actual RSA stuff was changed.


24 Feb
2012
FlickrAPI and OAuth

This summer Flickr will stop supporting their old authentication system and move fully to OAuth. I've gotten several questions about whether Python FlickrAPI will support that. No worries, it will. If you want it fast, clone the repository, implement OAuth (don't forget the unittests) and send me a pull request.


05 Aug
2011
Python-RSA version 3.0 released

Hacking, the analog way

Python-RSA version 3.0 has been released. A lot of things have been improved:

  • Much more secure

  • Following industrial standard PKCS#1; this means compatibility with OpenSSL

  • Ability to create and verify detached signatures

  • More elegant method of encrypting large files

  • Ability to save and load keys in PEM and DER format

  • Extensive documentation

  • More efficient key generation

Check the Python-RSA page for more information!

photo hosted on Flickr

Update: in the mean time 3.0.1 has been released, which removes an import of the "abc" module. This should make the code compatible with Python 2.5 again. All the documentation (including installation instructions) are the same as before.


10 Jul
2011
Added random padding to Python-RSA

Hacking, the analog way

I'm thrilled to let you know that I've just added encryption and decryption with random padding to my Python RSA module. The code has become much more secure than it was before, and also makes it capable of encrypting and decrypting binary data! This is a big step towards the practicality of the implementation.

I've implemented the padding according to the PKCS#1 standard. The RSA Algorithm page by DI Management explains how to do this very clearly; cudos to them!

Right now these changes are unreleased. I want to add signing and verifying signatures too before I do that. Maybe I'll deprecate the existing functions, as they are much less secure than all the new stuff I'm adding. If you want to try out what I've changed so far, feel free to grab a copy off my Mercurial repository.

photo hosted on Flickr

UPDATE 14:20: I've also implemented PKCS#1 style signatures and signature verification

UPDATE 2011-07-12: the implementation is compatible with OpenSSL, making it very likely compatible with other implementations of PKCS#1 version 1.5!


17 Jun
2011
Python vs. Java revisited

Python

Back in 2008 I wrote a small post about Python vs. Java. I've revamped it by adding more detail, and tossed in a primer on list comprehensions for free. I hope that once you've read that you're convinced that list comprehensions are a Good Thing.

And as an added bonus, I extended my website so that it can do syntax highlighting!

On to the article!


29 Apr
2011
Python serial comm and RTS pin

At my work I'm communicating with an embedded device via a serial-to-USB cable. It works fine, except for one little detail: when the "Ready To Send" (RTS) pin is high, the device shuts down. This is by design, but annoying nonetheless, since almost all serial communication software sets this pin to high. I've written a little Python program that opens the serial connection, sets the RTS pin to low, and shows all data read from it.

You can read the details after the break.


09 Jan
2011
Version 2.0 of RSA module

Hacking, the analog way

A guy named Dave sent me an email today, telling me that my RSA implementation was rather insecure due to the use of the "pickle" Python module. Fortunately I had some nice improvements by Barry Mead already in version control, eagerly waiting to be released. Well, Dave gave me a nice insentive to round up Barry's improvements and send version 2.0 into the world.

Please note that it's a backward-incompatible change. The RSA page shows how you can re-encrypt your data to be compatible (and more secure!) again.


20 Aug
2010
Extending Selenium from the client driver

Selenium is a web page testing tool that we use a lot at my work. It can be extended by supplying it with a user-extensions.js file.

We're extending our test application to include support for Selenium Grid. In this setup we can't supply that user-extensions.js file directly, it has to be injected via the client driver. However, this happens too late; Selenium doesn't notice our custom locator functions any more. This is a known issue, also noted by Ross Patterson. I've found a way to work around this limitation. Add the following to your user-extensions.js file such that it's executed after your additions have loaded:

  objectExtend(selenium.browserbot, PageBot.prototype);
selenium.browserbot._registerAllLocatorFunctions();

It seems to work just fine without any unwanted side-effects. If you tried this, please leave a comment to let us know how it worked out for you!


22 May
2010
Multiple instances of QApplication in one process

I'm working on a PyQt 4 application for my employer Chess. Of course I'm writing plenty of unit tests for it as well. Those tests have to be cleanly separated from each other, so every test that requires the entire application to be up and running should have its own instance. And this is where it gets tricky, as Qt (and thus PyQt) assumes that there will only be one application, and if that application quits the process quits.

In my unit tests, this is not the case. Every test can potentially start and stop an application, and the test suite will continue running. This caused a segmentation fault (a.k.a. segfault) the second time the QApplication closed.

After a lot of puzzling I found out that QtGui.qApp needs to refer to the running QApplication instance, and be set to None when the application has been closed. After I implemented this the segfaults went away.


This is my ApplicationTest mix-in class that I use for controlling my application lifespan. It contains a little trick to stop the application as soon as it has started:

  class MyApplicationClass(QtGui.QApplication):

  started = QtCore.pyqtSignal()

  def exec_(self):
      self.started.emit()
      return QtGui.QApplication.exec_()

class _ApplicationTest(object):
  '''Mix-in class that can start and stop an application.'''

  APP_CLASS = MyApplicationClass

  def __init__(self):
      self.app = None
      QtGui.qApp = None

  def create_app(self):
      '''Creates and returns a new application'''

      self.app = self.APP_CLASS([])
      QtGui.qApp = self.app
      return self.app

  def stop_app(self):
      '''Stops the application.'''

      if not self.app: return

      self.app.started.connect(self.app.quit,
              type=QtCore.Qt.QueuedConnection)
      self.app.exec_()

      QtGui.qApp = None
      self.app = None

10 Feb
2010
Song title fades in AfterEffects

I've been working on a video of our live performance in Winston Kingdom using Adobe AfterEffects. To show the title of a song at the start and end of the song, I used a script to automate the fading. Assign it as an expression to the opacity of the text layer and it'll automatically show for a few seconds at the in-point and out-point of the layer, including nice fades:

  fade_time = 1
show_time = 5
if (time < inPoint + fade_time) {
    linear(time, inPoint, inPoint + fade_time, 0, 100)
} else if (time < inPoint + fade_time + show_time) {
    100
} else if (time < inPoint + fade_time + show_time + fade_time) {
    linear(time, inPoint + fade_time + show_time, inPoint + fade_time + show_time + fade_time, 100, 0)
} else if (time < outPoint - fade_time - show_time - fade_time) {
    0
} else if (time < outPoint - fade_time - show_time) {
    linear(time, outPoint - fade_time - show_time - fade_time, outPoint - fade_time - show_time, 0, 100)
} else if (time < outPoint - fade_time) {
    100
} else if (time < outPoint) {
    linear(time, outPoint - fade_time, outPoint, 100, 0)
} else {
   0
}

fade_time is the duration of fade-in and fade-out, show_time determines how long the text layer is shown at 100% opacity, both are in seconds. You can make an animation preset by adding a "Solid Composite" effect, setting opacity=0% and assigning the expression to the "source opacity" property.


03 Feb
2010
Python Flickr API 1.4 released

The Python FlickrAPI kit has just been released. The new features are:

  • Using auth_callback=False when authentication is actually required now raises a FlickrError exception.

  • The implementation uses self.flickr_host so that subclasses can override the API URLs.

  • Support for short URLs was added.

Of course the Python FlickrAPI kit is completely dynamic, and so it always supports any new FlickrAPI functionality as soon as it is released by the Flickr team.

The new version can be found at https://stuvel.eu/projects/flickrapi or the Python Package Index at https://pypi.python.org/pypi/flickrapi


03 Oct
2009
Python Flickr API 1.3 released

Today I released version 1.3 of the Python Flickr API. It's clear that I've been busy with my study and my work, as the world hasn't seen a release in nearly a year. This is mostly due to the dynamic nature of the code - it automatically adapts to any new functionality that's added to Flickr, so no release is required then. This release adds some nifty new things:

  • Added functions to easily walk through sets and search results, querying Flickr no more than needed.

  • Uses the hashlib module, falling back to the md5 module when hashlib is unavailable.

  • Added locking token cache, in case a Flickr API key is used by multiple processes at the same time on the same machine (or shared filesystem)

  • Removed the deprecated fail_on_error parameter.

  • Implemented the auth_callback functionality.

Go to the Python Flickr API.


30 Jun
2009
Refusing to continue

I've seen this so many times, and thought about writing about it. Today I found some code that is too good an example to let it go. I'm talking about the refusal some people have to using the continue statement. Here is an example in some pseudo-code:

for value in list {
  if value.condition == True {
      do stuff
      do more stuff
  }
}

It looks like fine code, but personally I'd rather use:

for value in list {
  if value.condition == False: continue

  do stuff
  do more stuff
}

Read more about my coding philosophy after the break


22 Jan
2009
RSA version 1.3 released

A new version of my pure-Python RSA implementation has been released. The one major change is improved compatibility with Windows.

Download EGG files for Python 2.4, Python 2.5 or the source package at the Python Package index.


15 Nov
2008
Python Flickr API 1.2 released

Today I released the Python Flickr API version 1.2. Originally the Python interface was made by Beej, and since it's seen many an improvement. Here's what's new in this version:

  • Added ElementTree support for Python 2.4.

  • Made ElementTree the default response parser.

  • The upload and replace methods now take a format parameter.

  • Removed deprecated methods.

  • Upload and replace methods no longer report progress on their callback regarding the HTTP headers.


06 Nov
2008
Hacking in Fallout 3

Fallout 3 is my latest addiction. It's a great game by the makers of Oblivion. Part of the game is the hacking of computer terminals. It is basically a Master Mind game, but then with letters. You click on a word that shows up in a memory dump of the terminal you are trying to hack, and the terminal tells you how many letters are correct and in the correct place. Of course, a real hacker would write a program to solve the puzzle. So, here it is:

  #!/usr/bin/env python

known = {
    'detector': 1,
}

words = '''
detector carpeted declared detailed dramatic devoured demanded
sinister disposal revealed messages released remained defended
peaceful designed prisoner
'''

words = set(words.split())

def dist(a, b):
    return sum(la == lb for (la, lb) in zip(a, b))

for k in known:
    for w in set(words):
        if dist(k, w) != known[k]:
            words.discard(w)
print "Try any of:", ', '.join(words)

The variable words should be filled with all the words you see on the terminal screen. The known variable should contain a dictionary with the known information you got from the terminal. For example, if you pick those:

  • detector: 1/8 correct

  • demanded: 2/8 correct

then the known variable should be:

  known = {
    'detector': 1,
    'demanded': 2,
}

If you then run the program (using python fallout.py if you saved the script as fallout.py) it will tell you to try "carpeted" or "messages" as your next guess.

The script requires Python, which is installed by default on most Linux and MacOS systems and should be manually installed on Windows.

update: I have automated the hacking so you can easily use this website to make the sometimes difficult hacking a lot easier.


03 Nov
2008
Java does not support Unicode, Python does

Sometimes Java just amazes me. As a friend of mine posted, Java does not support Unicode. Of course, my favourite language Python has a much friendlier approach to Unicode:

>>> s = '⿱𠂉乙'.decode('utf-8')
>>> s
u'\u2ff1\U00020089\u4e59'
>>> s[1]
u'\U00020089'
>>> print s[1]
𠂉

Perhaps,. people will some day start to understand that statically typed languages are not a guarantee of a bug-free application. In the end, it's more important that a language does what it is supposed to do.


06 Jun
2008
Pure-Python RSA module version 1.2 released under EUPL

By popular demand, we've re-licensed our pure-Python RSA module under the European Union Public Licence (EUPL). This new licence is the first open source license to be released by an international governing body.

RSA version 1.2 is code-wise the same as the previous version - the only difference is the new license. It is also still compatible the GPL. Actually, the module is now dual-licensed, under both the GPL and the EUPL. Take your pick, and use which one suits you best.


10 Apr
2008
Python vs. Java

I've updated this article on 2011-06-15. Comments from back in 2008 are referring to the original version (which cut some corners and was a little too short).

One of the major reasons my company prefers Java over Python, is Java's static type declarations (and all the benefits that follow). If only Python had that, my life would be so much nicer. Here is an example of code I have to work with, and an example of what the Python equivalent would look like.

The background of the code: We have a list of "identifiers" that can identify people who want to use a parking garage. These can be NFC cards, credit cards, and bar codes. We want to filter the list of known identifiers and return only those of type "barcode".

The code in Java:

private List<Identifier> barcodeTypeOnly(List<Identifier> ids)
{
    final List<Identifier> barcodes = new ArrayList<Identifier>();

    for (Identifier id : ids) {
        if (id.getType().equals(Identifier.TYPEBARCODE)) {
            barcodes.add(id);
        }
    }

    return barcodes;
}

What it would look like in Python:

ids = the list of identifiers

barcodes = [identifier for identifier in ids
            if identifier.type == Identifier.TYPEBARCODE]

You could write that last one on a single line, but for clarity I wrote it like this. I understand that the list comprehension construction may not be instantly recognisable, so I wrote a little explanation after the break.