TensorFlow/PyTorch Framework Fundamentals Questions
Practical knowledge of a major deep learning framework. Includes understanding tensors, operations, building neural network layers, constructing models, and training loops. Ability to read and modify existing code in these frameworks. Knowledge of how to work with pre-built layers and models.
HardTechnical
48 practiced
Outline how to implement pipeline parallelism for a very large model using torch.distributed.rpc or torch.distributed.pipeline.sync. Describe how to partition the model across worker ranks, schedule forward and backward passes, manage micro-batches to keep the pipeline filled, handle parameter synchronization and checkpointing, and measure the latency vs throughput trade-offs.
HardTechnical
48 practiced
Provide step-by-step code and explanation to convert a PyTorch model to ONNX and then import it into TensorFlow using onnx-tf or tf2onnx. Address handling dynamic axes, unsupported ops and operator version mismatches, and how to validate functional equivalence between the original PyTorch and the converted TensorFlow model.
HardTechnical
40 practiced
For training a large Transformer model, propose concrete strategies to optimize memory and compute: mixed precision, activation checkpointing (activation recomputation), model parallelism, and optimizer state sharding (ZeRO). Compare practical implementation differences between TensorFlow (activation_checkpointing, XLA) and PyTorch (torch.utils.checkpoint, DeepSpeed, FairScale).
MediumTechnical
37 practiced
Implement a custom TensorFlow Keras layer in Python that computes y = alpha * x + beta, where alpha and beta are trainable variables with specified initializers and optional regularizers. Your layer must implement build, call, and get_config to be serializable so models using it can be saved and loaded.
MediumTechnical
42 practiced
Discuss trade-offs between eager execution and graph-mode in TensorFlow. Provide practical examples where tf.function increases throughput and where eager mode simplifies debugging and supports dynamic control flow. Mention retracing costs and how to mitigate them.
Unlock Full Question Bank
Get access to hundreds of TensorFlow/PyTorch Framework Fundamentals interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.