基于Theano的可扩展深度学习框架：deepy
jopen
9年前
基于Theano的可扩展深度学习框架deepy。
Dependencies
- Python 2.7 (Better on Linux)
- numpy
- theano
- scipy for L-BFGS and CG optimization
Clean interface
# MNIST Multi-layer model with dropout. from deepy.dataset import MnistDataset, MiniBatches from deepy.networks import NeuralClassifier from deepy.layers import Dense, Softmax, Dropout from deepy.trainers import MomentumTrainer, LearningRateAnnealer model = NeuralClassifier(input_dim=28*28) model.stack(Dense(256, 'relu'), Dropout(0.2), Dense(256, 'relu'), Dropout(0.2), Dense(10, 'linear'), Softmax()) trainer = MomentumTrainer(model) annealer = LearningRateAnnealer(trainer) mnist = MiniBatches(MnistDataset(), batch_size=20) trainer.run(mnist, controllers=[annealer])
Examples
Enviroment setting
- CPU
source bin/cpu_env.sh
- GPU
source bin/gpu_env.sh
MNIST Handwriting task
- Simple MLP
python experiments/mnist/mlp.py
- MLP with dropout
python experiments/mnist/mlp_dropout.py
- MLP with PReLU and dropout
python experiments/mnist/mlp_prelu_dropout.py
- Deep convolution
python experiments/mnist/deep_convolution.py
- Elastic distortion
python experiments/mnist/mlp_elastic_distortion.py
- Recurrent visual attention model </li> </ul>
python experiments/attention_models/baseline.py
Language model
Penn Treebank benchmark
- Baseline RNNLM (Full-output layer)
python experiments/lm/baseline_rnnlm.py
- Class-based RNNLM
python experiments/lm/class_based_rnnlm.py
- LSTM based LM (Full-output layer)
python experiments/lm/lstm_rnnlm.py
Char-based language models
- Char-based LM with LSTM
python experiments/lm/char_lstm.py
- Char-based LM with Deep RNN
python experiments/lm/char_rnn.py
Deep Q learning
- Start server
pip install Flask-SocketIO python experiments/deep_qlearning/server.py
- Open this address in browser
http://localhost:5003
Auto encoders
- Recurrent NN based auto-encoder
python experiments/auto_encoders/rnn_auto_encoder.py
- Recursive auto-encoder
python experiments/auto_encoders/recursive_auto_encoder.py
Train with CG and L-BFGS
- CG
python experiments/scipy_training/mnist_cg.py
- L-BFGS
python experiments/scipy_training/mnist_lbfgs.py
Other experiments
DRAW
See https://github.com/uaca/deepy-draw
# Train the model python mnist_training.py # Create animation python animation.py experiments/draw/mnist1.gz
Highway networks
python experiments/highway_networks/mnist_baseline.py python experiments/highway_networks/mnist_highway.py
Effect of different initialization schemes
python experiments/initialization_schemes/gaussian.py python experiments/initialization_schemes/uniform.py python experiments/initialization_schemes/xavier_glorot.py python experiments/initialization_schemes/kaiming_he.py
Other features
- Auto gradient correction
Sorry for that deepy is not well documented currently, but the framework is designed in the spirit of simplicity and readability. This will be improved if someone requires.