Learning to Strike for Robotic Table Tennis

Abstract

Motivated by recent advances in robotic systems’ ability to interact with dynamic environments, we study autonomous robotic table tennis in this paper. We design a high speed robotic arm to play table tennis, and develop a software system that predicts the incoming ball’s trajectory and learns to control the robot arm for striking the ball to the opponent side of the table. Utilizing the idea of residual physics, we develop a residual predictor to predict a striking position for the incoming ball using sparse observation of the ball’s starting trajectory. We control the arm to the contact-ready position, and then use a neural network to output the striking velocity to hit the ball. The neural network is trained through experience using a regression-based learning method. Simulation results in a robotic simulator demonstrate the superior performance, benefiting from efficient learning.

Highlights

A residual learning scheme that predicts the incoming ball’s landing/striking position from just two position samples of its early trajectory — correcting a physics-based estimate for real-world effects (restitution, friction, air resistance) rather than modeling them explicitly.
A random forest regressor that learns the striking velocity needed to return the ball to a target landing position, trained from experience rather than hand-tuned control.
A custom-built 6-DOF high-speed robotic arm, since most commercial arms are too speed-limited for table tennis.
98.5% interception success (985/1000 simulated throws) with the residual predictor, versus consistent failure using physics-only trajectory prediction alone.
Both learned components run in a few milliseconds — under 16 ms combined — leaving ample time for motion planning and control.

The System & Problem

Commercial robotic arms are designed for safe, limited-speed operation, making most of them unsuitable for table tennis. We therefore design a custom 6-DOF arm: joints J₁–J₄ use Xiaomi Cybergear motors (continuous rotation, up to 296 rpm, 12 N·m peak torque), while J₅ and J₆ use RDS5160 servos (0–270°, up to 461°/s, 6.87 N·m peak torque) placed along the paddle’s roll, pitch, and yaw axes for precise orientation control.

Design of the custom 6-DOF robotic arm, showing the Xiaomi Cybergear motors, RDS5160 servo motors, and paddle end-effector

Fig. 1 — The custom high-speed 6-DOF arm. J₁–J₄ use Xiaomi Cybergear motors; J₅–J₆ use RDS5160 servos and are aligned with the paddle’s roll/pitch/yaw for precise orientation.

Given the first two camera-observed positions of a thrown ball, p₀ and p₁, the goal is to compute joint angular velocities that strike the ball so it lands at a desired location p_L on the opponent’s side — with the striking point p_s constrained to a fixed plane in front of the robot, and a pre-strike paddle orientation θ_d.

Setup diagram: a camera observes the ball's early trajectory, the robot arm strikes it toward target zones on the opponent's side of the table

Fig. 2 — The table tennis setup. A camera observes the ball’s first two positions p₀, p₁; the robot strikes the ball toward a target landing position p_L in the opponent’s target zones.

Method

The system has three modules: a Striking Point Estimation module predicts where to intercept the ball, a Contact Ready module moves the paddle there, and a Ball Strike module predicts the joint velocities needed to hit it toward the desired landing spot.

System architecture: striking point estimation module, contact ready module, and ball strike module, with physics-based and neural-network submodules

Fig. 3 — System architecture. The physics-based ideal trajectory (purple) is corrected by a learned residual (green) to estimate the striking point p_s, which feeds a motion planner (orange) and a learned striking-velocity planner (pink).

Striking Point Estimation: Residual Learning

From p₀ and p₁, an initial velocity v₀ gives an ideal projectile-motion trajectory P_t. But real trajectories deviate from this ideal due to the table’s coefficient of restitution, friction, and air resistance — effects that are hard to model explicitly. Rather than modeling them, we learn the deviation δ directly: a shallow feedforward network (two linear layers, ReLU then TanH) takes p₀, v₀, the distance to the robot, and the ideal striking point p_k, and predicts δ, so the actual striking point is p_s = P_t + δ. The shallow architecture keeps inference to about 4.3 ms.

Residual predictor architecture (two linear layers with ReLU and TanH activations) and an illustration of the ideal versus actual ball trajectory

Fig. 4 — The residual predictor: a two-layer network mapping (p₀, v₀, d, p_k) to the residual δ between the ideal (magenta) and actual (green) trajectory.

Contact Ready & Motion Planning

Once p_s and the pre-strike orientation θ_d are known, an RRT-Connect motion planner computes the joint trajectory that brings the paddle from its current pose to the target pose, executed by the arm’s joint position controller.

Ball Strike: Learning the Striking Velocity

With the paddle positioned at p_s, the joint velocities ω₄, ω₅, ω₆ (roll/pitch/yaw — sufficient to direct the ball anywhere) are predicted as a function of v₀, p_s, and the desired landing position p_L. We model this mapping with a random forest regressor, trained on data collected by hitting the ball with random joint velocities and recording where it lands; a PID controller then executes the predicted velocities at the moment of contact.

Results

Validated in a PyBullet simulation matching standard table tennis dimensions and ball properties (20 mm radius, 2.7 g), with a restitution coefficient of 0.75 and modeled table friction and air resistance.

Residual Predictor

Across 1000 thrown-ball trials, the residual predictor outperforms every baseline regressor tested on MSE, MAE, and R² for estimating δ.

Model	MSE (y)	MAE (y)	R² (y)	MSE (z)	MAE (z)	R² (z)
SVR	0.0011	0.0274	−4.29	0.0069	0.0683	0.775
Decision Tree	0.0004	0.0074	−0.94	0.0069	0.0483	0.777
Random Forest (20)	0.00023	0.00688	−0.05	0.00409	0.0387	0.869
KNN	0.0002	0.0061	−0.33	0.0051	0.0475	0.835
Ours	0.0001	0.0053	0.199	0.0041	0.0346	0.888

The Y-axis R² is comparatively modest, but the residual there is small in absolute terms next to the Z-axis residual — qualitatively, our model still reliably locates the collision point.

Motion Planning

Among eight classical motion planners benchmarked over 1000 random target points, RRT-Connect had both the lowest mean planning time and the tightest interquartile range, motivating its use in the contact-ready module.

Boxplot of planning times for RRT, PRM, RRT*, RRT-Connect, TRRT, BiTRRT, PDST, and EST

Fig. 5 — Planning time comparison across motion planners; RRT-Connect (bold) offers the most consistent, lowest planning time.

Striking Velocity Planner

Repeating the striking-velocity experiment 1000 times with varied throw velocities and target landing positions, the Random Forest regressor with 160 estimators achieved the best R² across all three rotation axes tested.

Model	Roll			Pitch			Yaw
	R²	MAE	MSE	R²	MAE	MSE	R²	MAE	MSE
Decision Tree	0.681	2.49	20.3	0.702	4.27	38.5	0.169	2.13	8.06
KNN	0.796	2.32	13.0	0.652	4.33	45.0	0.662	1.37	3.27
SVR	0.800	2.70	12.7	0.665	4.15	43.3	0.596	1.52	3.92
Random Forest (160, ours)	0.887	1.54	7.21	0.689	4.33	40.2	0.687	1.35	3.04

Two example target zones showing intended hitting point (red X) and where the ball actually landed across trials (green X marks clustered nearby)

Fig. 6 — Qualitative accuracy of the striking velocity planner: intended target (red ×) versus actual landing positions (green ×) across repeated trials.

End-to-End Performance

Across 1000 randomly thrown balls, the full system — residual predictor plus learned striking velocity — successfully intercepted 985, a 98.5% success rate. Without the residual predictor, relying on the physics-only trajectory alone, the system consistently failed to intercept the ball, since each bounce alters the trajectory in ways the ideal model cannot capture.

Paddle velocity over the four operational stages A (estimate/plan), B (move to contact-ready), C (wait for ball), D (execute strike), with corresponding arm poses below

Fig. 7 — Paddle velocity across the four operational stages: (A) estimate the striking point and plan, (B) move to the contact-ready pose, (C) wait for the ball, (D) execute the learned strike.

Fig. 8 — Inference times: the residual predictor (median 4.3 ms) and striking velocity planner (median 4.2 ms) together add well under 16 ms, leaving ample budget for motion planning and control.

Limitations & Future Work

This work validates the approach entirely in a PyBullet simulation, assuming the first two ball positions come from an upstream perception system; a real robot experiment is beyond the current paper’s scope. Future work includes integrating a high-speed 3D perception system for real-time ball tracking on the physical arm, building a real-time control stack for low-latency contact-ready and strike execution, adapting the striking-point estimator to real sensor noise, and fine-tuning motion planning and control for the hardware’s constraints.

Conclusion

We presented a learning-from-demonstration approach to robotic table tennis built from three modules: striking point estimation, contact-ready motion, and ball striking. A residual learning scheme corrects a physics-based trajectory estimate using only the ball’s first two observed positions, and a random forest regressor learns the striking velocity needed to direct the ball to a target landing position. Compared to reinforcement-learning approaches that require extensive, potentially unsafe real-world trial-and-error, this sample-efficient formulation reaches 98.5% interception success in simulation while keeping both learned components fast enough — a few milliseconds each — for real-time play.