DeepSpeed
  • Training Setup
  • Inference Setup
  • Training API
  • Inference API
  • Model Checkpointing
  • Activation Checkpointing
  • ZeRO
  • Mixture of Experts (MoE)
  • Transformer Kernels
  • Pipeline Parallelism
  • Optimizers
  • Learning Rate Schedulers
  • Flops Profiler
  • Autotuning
  • Memory Requirements
  • Monitoring
DeepSpeed
  • Overview: module code

All modules for which code is available

  • deepspeed
    • deepspeed.checkpoint.utils
    • deepspeed.comm.comm
    • deepspeed.inference.config
    • deepspeed.moe.layer
    • deepspeed.monitor.config
    • deepspeed.ops.adam.cpu_adam
    • deepspeed.ops.adam.fused_adam
    • deepspeed.ops.lamb.fused_lamb
    • deepspeed.ops.transformer.transformer
    • deepspeed.profiling.flops_profiler.profiler
    • deepspeed.runtime.fp16.onebit.adam
    • deepspeed.runtime.fp16.onebit.lamb
    • deepspeed.runtime.fp16.onebit.zoadam
    • deepspeed.runtime.lr_schedules
    • deepspeed.runtime.pipe.engine
    • deepspeed.runtime.pipe.module
    • deepspeed.runtime.pipe.schedule
    • deepspeed.runtime.pipe.topology
    • deepspeed.runtime.zero.config
    • deepspeed.runtime.zero.offload_config
    • deepspeed.runtime.zero.stage3
    • deepspeed.runtime.zero.stage_1_and_2
    • deepspeed.utils.tensor_fragment
    • deepspeed.utils.zero_to_fp32

© Copyright 2020, Microsoft.

Built with Sphinx using a theme provided by Read the Docs.