Chainer v2 and the future dev planChainer Meetup #05 @ Microsoft Japan, June 10, 2017

Seiya Tokui @ Preferred Networks

Chainer v2.0.0

has been released onJune 1, 2017 ✌

Thank you for all contributions!!!

Chainer v2

First time to break backward compatibility since v1.5.0

• Cleaner APIs

• Improved internal design (w/ memory reduction)

• Better PyCharm support

2015/11

https://docs.chainer.org/en/stable/upgrade.html

For contributors

Contribution Guide has been updated! Please read it again!

https://docs.chainer.org/en/stable/contribution.html

For users

We apologize that we could not fulfill the compatibility policy

in v2.0.0 (in particular, some changes are made without

explicit deprecation steps).

We have also updated the compatibility policy.

https://docs.chainer.org/en/stable/compatibility.html

Important changes that affect user code

• CuPy is separated into a distinct repository/package

• Thread-local configuration chainer.config is introduced

• Training mode: chainer.config.train

• “volatile” is replaced by chainer.config.enable_backprop

• Interfaces of some links are changed

• Many deprecated classes/methods/functions are removed

Installation tips for GPU users

CuPy is now a separate package.

• If you have an old chainer, uninstall it first to clear CuPy:

pip uninstall chainer (repeat it until an error occurs)

• Then, install CuPy & Chainer by pip install cupy chainer

• If you are working with sudo, be careful that the environment

variables are not passed by default through sudo. If the nvcc

command is not found through PATH variable, cupy setup script fails.

New features (selected)

Aggressive buffer release

• Functions that do not require some of inputs for backprop

can release the array grabbed by the input nodes

• E.g.: F.relu and `+` can release the inputs

• It required refactoring of object relationships around

Variable and Function (so we introduced VariableNode)

33% memory reduction for ResNet!!

New features (selected)

Fine-grained optimization control

• Optimizer.setup sets up update_rule to each parameter

• Each update rule can have its own hyperparameter

(e.g. learning rate)

• Each update rule can have its own hook function

(e.g. weight decay)

• Each update rule can be disabled (layer freezing)

New features (also included in v1)

Advanced indexing

• Variable and cupy.ndarray support advanced indexing

• Advanced indexing: indexing by integer index arrays and

Boolean mask arrays

• Variable’s advanced indexing also supports backprop

Future development plan

The main repository is moved to

https://github.com/chainer/chainer

Old path (pfnet/chainer) is automatically redirected to new

one (both on web and on git)

Related packages will also be developed under

chainer organization

New release cycle

No minor, 3 pre-releases, each release taking 4 weeks.

Versioned branches keep compatibility.

Developed inmaster branch

Developed inversioned branch

(v2, v3, v4, …)

Future plan

• Differentiable backprop (a.k.a. backprop of backprop, double

backprop, higher order differentiation, etc.)

• Make a new-style Function whose backward() is implemented on

Function interface instead of ndarray

• Some more refactoring of object relationships around Variable

could be needed

• Documentation improvements, refactoring of tests

• Graph visualization toolkit?

• and more…

Discussions on the development

The main channel of the core dev team has moved to the

Chainer Slack team (request an invitation at bit.ly/chainer-slack)

• #develop for Chainer

• #develop-cupy for CuPy

These channels are public, so anyone can watch what is

going on in our daily development. Discussions on the

development are also welcome!