Dynamic images and image views

The GIL extension called dynamic_image allows for images, image views or any GIL constructs to have their parameters defined at run time.

The color space, channel depth, channel ordering, and interleaved/planar structure of an image are defined by the type of its template argument, which makes them compile-time bound. Often some of these parameters are available only at run time. Consider, for example, writing a module that opens the image at a given file path, rotates it and saves it back in its original color space and channel depth. How can we possibly write this using our generic image? What type is the image loading code supposed to return?

Here is an example:

#include <boost/gil/extension/dynamic_image/dynamic_image_all.hpp>
using namespace boost;

#define ASSERT_SAME(A,B) static_assert(is_same< A,B >::value, "")

// Define the set of allowed images
typedef mpl::vector<rgb8_image_t, cmyk16_planar_image_t> my_images_t;

// Create any_image class (or any_image_view) class
typedef any_image<my_images_t> my_any_image_t;

// Associated view types are available (equivalent to the ones in image_t)
typedef any_image_view<mpl::vector2<rgb8_view_t,  cmyk16_planar_view_t > > AV;
ASSERT_SAME(my_any_image_t::view_t, AV);

typedef any_image_view<mpl::vector2<rgb8c_view_t, cmyk16c_planar_view_t> > CAV;
ASSERT_SAME(my_any_image_t::const_view_t, CAV);
ASSERT_SAME(my_any_image_t::const_view_t, my_any_image_t::view_t::const_t);

typedef any_image_view<mpl::vector2<rgb8_step_view_t, cmyk16_planar_step_view_t> > SAV;
ASSERT_SAME(typename dynamic_x_step_type<my_any_image_t::view_t>::type, SAV);

// Assign it a concrete image at run time:
my_any_image_t myImg = my_any_image_t(rgb8_image_t(100,100));

// Change it to another at run time. The previous image gets destroyed
myImg = cmyk16_planar_image_t(200,100);

// Assigning to an image not in the allowed set throws an exception
myImg = gray8_image_t();        // will throw std::bad_cast

The any_image and any_image_view subclass from GIL variant class, which breaks down the instantiated type into a non-templated underlying base type and a unique instantiation type identifier. The underlying base instance is represented as a block of bytes. The block is large enough to hold the largest of the specified types.

GIL variant is similar to boost::variant in spirit (hence we borrow the name from there) but it differs in several ways from the current boost implementation. Perhaps the biggest difference is that GIL variant always takes a single argument, which is a model of MPL Random Access Sequence enumerating the allowed types. Having a single interface allows GIL variant to be used easier in generic code. Synopsis:

template <typename Types>    // models MPL Random Access Container
class variant
{
  ...         _bits;
  std::size_t _index;
public:
  typedef Types types_t;

  variant();
  variant(const variant& v);
  virtual ~variant();

  variant& operator=(const variant& v);
  template <typename TS> friend bool operator==(const variant<TS>& x, const variant<TS>& y);
  template <typename TS> friend bool operator!=(const variant<TS>& x, const variant<TS>& y);

  // Construct/assign to type T. Throws std::bad_cast if T is not in Types
  template <typename T> explicit variant(const T& obj);
  template <typename T> variant& operator=(const T& obj);

  // Construct/assign by swapping T with its current instance. Only possible if they are swappable
  template <typename T> explicit variant(T& obj, bool do_swap);
  template <typename T> void move_in(T& obj);

  template <typename T> static bool has_type();

  template <typename T> const T& _dynamic_cast() const;
  template <typename T>       T& _dynamic_cast();

  template <typename T> bool current_type_is() const;
};

template <typename UOP, typename Types>
 UOP::result_type apply_operation(variant<Types>& v, UOP op);
template <typename UOP, typename Types>
 UOP::result_type apply_operation(const variant<Types>& v, UOP op);

template <typename BOP, typename Types1, typename Types2>
 BOP::result_type apply_operation(      variant<Types1>& v1,       variant<Types2>& v2, UOP op);

template <typename BOP, typename Types1, typename Types2>
 BOP::result_type apply_operation(const variant<Types1>& v1,       variant<Types2>& v2, UOP op);

template <typename BOP, typename Types1, typename Types2>
 BOP::result_type apply_operation(const variant<Types1>& v1, const variant<Types2>& v2, UOP op);

GIL any_image_view and any_image are subclasses of variant:

template <typename ImageViewTypes>
class any_image_view : public variant<ImageViewTypes>
{
public:
  typedef ... const_t; // immutable equivalent of this
  typedef std::ptrdiff_t x_coord_t;
  typedef std::ptrdiff_t y_coord_t;
  typedef point<std::ptrdiff_t> point_t;
  using size_type = std::size_t;

  any_image_view();
  template <typename T> explicit any_image_view(const T& obj);
  any_image_view(const any_image_view& v);

  template <typename T> any_image_view& operator=(const T& obj);
  any_image_view&                       operator=(const any_image_view& v);

  // parameters of the currently instantiated view
  std::size_t num_channels()  const;
  point_t     dimensions()    const;
  size_type   size()          const;
  x_coord_t   width()         const;
  y_coord_t   height()        const;
};

template <typename ImageTypes>
class any_image : public variant<ImageTypes>
{
  typedef variant<ImageTypes> parent_t;
public:
  typedef ... const_view_t;
  typedef ... view_t;
  typedef std::ptrdiff_t x_coord_t;
  typedef std::ptrdiff_t y_coord_t;
  typedef point<std::ptrdiff_t> point_t;

  any_image();
  template <typename T> explicit any_image(const T& obj);
  template <typename T> explicit any_image(T& obj, bool do_swap);
  any_image(const any_image& v);

  template <typename T> any_image& operator=(const T& obj);
  any_image&                       operator=(const any_image& v);

  void recreate(const point_t& dims, unsigned alignment=1);
  void recreate(x_coord_t width, y_coord_t height, unsigned alignment=1);

  std::size_t num_channels()  const;
  point_t     dimensions()    const;
  x_coord_t   width()         const;
  y_coord_t   height()        const;
};

Operations are invoked on variants via apply_operation passing a function object to perform the operation. The code for every allowed type in the variant is instantiated and the appropriate instantiation is selected via a switch statement. Since image view algorithms typically have time complexity at least linear on the number of pixels, the single switch statement of image view variant adds practically no measurable performance overhead compared to templated image views.

Variants behave like the underlying type. Their copy constructor will invoke the copy constructor of the underlying instance. Equality operator will check if the two instances are of the same type and then invoke their operator==, etc. The default constructor of a variant will default-construct the first type. That means that any_image_view has shallow default-constructor, copy-constructor, assignment and equality comparison, whereas any_image has deep ones.

It is important to note that even though any_image_view and any_image resemble the static image_view and image, they do not model the full requirements of ImageViewConcept and ImageConcept. In particular they don’t provide access to the pixels. There is no “any_pixel” or “any_pixel_iterator” in GIL. Such constructs could be provided via the variant mechanism, but doing so would result in inefficient algorithms, since the type resolution would have to be performed per pixel. Image-level algorithms should be implemented via apply_operation. That said, many common operations are shared between the static and dynamic types. In addition, all of the image view transformations and many STL-like image view algorithms have overloads operating on any_image_view, as illustrated with copy_pixels:

rgb8_view_t v1(...);  // concrete image view
bgr8_view_t v2(...);  // concrete image view compatible with v1 and of the same size
any_image_view<Types>  av(...);  // run-time specified image view

// Copies the pixels from v1 into v2.
// If the pixels are incompatible triggers compile error
copy_pixels(v1,v2);

// The source or destination (or both) may be run-time instantiated.
// If they happen to be incompatible, throws std::bad_cast
copy_pixels(v1, av);
copy_pixels(av, v2);
copy_pixels(av, av);

By having algorithm overloads supporting dynamic constructs, we create a base upon which it is possible to write algorithms that can work with either compile-time or runtime images or views. The following code, for example, uses the GIL I/O extension to turn an image on disk upside down:

#include <boost\gil\extension\io\jpeg_dynamic_io.hpp>

template <typename Image>    // Could be rgb8_image_t or any_image<...>
void save_180rot(const std::string& file_name)
{
  Image img;
  jpeg_read_image(file_name, img);
  jpeg_write_view(file_name, rotated180_view(view(img)));
}

It can be instantiated with either a compile-time or a runtime image because all functions it uses have overloads taking runtime constructs. For example, here is how rotated180_view is implemented:

// implementation using templated view
template <typename View>
typename dynamic_xy_step_type<View>::type rotated180_view(const View& src) { ... }

namespace detail
{
  // the function, wrapped inside a function object
  template <typename Result> struct rotated180_view_fn
  {
      typedef Result result_type;
      template <typename View> result_type operator()(const View& src) const
{
          return result_type(rotated180_view(src));
      }
  };
}

// overloading of the function using variant. Takes and returns run-time bound view.
// The returned view has a dynamic step
template <typename ViewTypes> inline // Models MPL Random Access Container of models of ImageViewConcept
typename dynamic_xy_step_type<any_image_view<ViewTypes> >::type rotated180_view(const any_image_view<ViewTypes>& src)
{
  return apply_operation(src,detail::rotated180_view_fn<typename dynamic_xy_step_type<any_image_view<ViewTypes> >::type>());
}

Variants should be used with caution (especially algorithms that take more than one variant) because they instantiate the algorithm for every possible model that the variant can take. This can take a toll on compile time and executable size. Despite these limitations, variant is a powerful technique that allows us to combine the speed of compile-time resolution with the flexibility of run-time resolution. It allows us to treat images of different parameters uniformly as a collection and store them in the same container.