I am developing an API in Unix environment for virtual machines. I have to store some information in a table about virtual machines.

Currently I am using python dictionary of virtual machine objects and storing the same in pickle.

I would like to know about other best ways(if any) to store the data in command line API's. Any suggestion would be helpful.

  • 1
    If you are working with YAML you might have a look at ruamel.yaml. This question was one of the reason I finally started that solving some of the PyYAML issues in that implementation. – Anthon Nov 24 '14 at 13:12
  • I appreciate your effort for the implementation. And I am very glad that this question is one of the reason for that. Sure, I will use this module for YAML. – Dany Nov 27 '14 at 7:15

For an application used by others you should probably not use pickle as it is not save to load pickle data from untrusted sources (alternatively you would have to make 100% sure the pickled data could not be changed on disc between writing and reading).

If the data ever needs modifying by users, you should have a look at writing it out using YAML. A Python implementation is available via PyYAML, but an emitter with better readable output would be pyaml.

Alternatives are JSON and with more work XML.

All of these require you to do translation to and from internal types to the types supported by the storage format (dictionaries/maps, lists, primitive types (ints, strings, boolean)).

YAML also support user defined types with the same security issues as pickled Python data, but at least allows some safe loading, something that pickle doesn't support. YAML, through the explicit user types has less problems loading data after changing your source layout (renaming modules, moving classes) as you can experience with pickle.

  • YAML is a good choice. It is similar to JSON, but slightly more powerful. Pickling (and serialization in general) is a bad choice for storing persistent config data. For one thing, it is not human readable. – Faheem Mitha Nov 9 '14 at 10:49
  • @FaheemMitha It is possible to pickle in text form (Protocol version 0), that makes it similar to XML: having the inefficiencies of ASCII combined with the unreadability of binary. – Anthon Nov 9 '14 at 10:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.