Fastest way to serialize an array in Python

To kick start the discussion, let’s first assume that we have the following 2-D numpy array: import numpy as np matrix = np.random.random((10000, 384)).astype(np.float32) Approaches tried The built-in pickle In Python, the built-in pickle module is often used to serialize/deserialize data. Here’s how it works: import pickle pickled_dumped = pickle.dumps(matrix, protocol=pickle.HIGHEST_PROTOCOL) pickle.loads(pickled_dumped) MessagePack MessagePack is known to a fast and compact serialization format. But it can’t serialize a numpy array directly, so let’s first assume that we’ll use a nested Python list instead.

Be careful of shared array

In Go, when you take a slice of an existing slice, you get two slices sharing the same underlying array. package main import "fmt" func main() { s1 := []int{1, 2, 3} // A slice of s1 => {2, 3} s2 := s1[1:3] // The 0th element of s2 is the 1th element of s1 s2[0] = 42 fmt.Println(s1) // [1 42 3] fmt.Println(s2) // [42 3] } And we can append items to slices with the built-in function append.

what you need to know about aws cloudformation

CloudFormattion is free to use; You supply a template and any required parameters to CloudFormation to create or update a stack; A template is a JSON-formatted text file (example); Parameter can be used to specify sensitive information which you don’t want to store in the template; If not all resources are created or updated successfully, CloudFormation will rollback every change;

define and run cython function with ipython

I learned this trick in the book Cython: A Guide for Python Programmer. After having both Cython and IPython installed, we can load the cythonmagic extension with: In [1]: %load_ext cythonmagic Now we have 3 extra magic functions in the session: %%cython will cythonize and compile contents of the code cell. In [4]: %%cython ...: def sum_up(*numbers): ...: cdef int s = 0 ...: for n in numbers: .