Residual DDPG + Computed Torque Control for a Cable-Driven Lower-Limb Rehabilitation Robot

This M.Sc. research investigates whether a bounded learned residual can improve a nominal nonlinear model-based controller for a cable-driven lower-limb rehabilitation mechanism when the plant is affected by parameter mismatch and external disturbances. The controller keeps Computed Torque Control (CTC) as the primary tracking layer and uses a Deep Deterministic Policy Gradient (DDPG) actor only to supply a limited corrective torque.

The study is deliberately broader than a Cartesian tracking comparison. Because cable-driven mechanisms are subject to unilateral actuation, the evaluation also examines joint limits, algebraic nonnegative-tension demand, sampled workspace feasibility, cable-Jacobian rank and conditioning, disturbance-seed robustness, interval-wise behavior, nearby initial conditions, and one alternate trajectory.

Research code: the working thesis repository is private; a curated research-code package can be shared on request.

The page uses a deliberately small portfolio figure set and reports the broader workspace, conditioning, case-matrix, seed, interval, residual-authority, and alternate-trajectory evidence in text and tables so that the scientific scope remains visible without reproducing the full manuscript.

Research problem and motivation

Computed torque control can provide accurate nonlinear tracking when the nominal dynamic model closely matches the plant. In a rehabilitation-oriented mechanism, however, masses, inertias, segment properties, attachment conditions, and external loads can differ from the model used by the controller. Exact inverse-dynamics cancellation then degrades, so the closed-loop system must cope with the remaining mismatch.

A cable-driven robot adds another complication: a Cartesian point may be kinematically reachable while still being problematic for the chosen cable routing because cables can pull but cannot push. Tracking performance therefore has to be interpreted together with joint-coordinate limits, cable-force demand, and the conditioning of the cable Jacobian.

Hybrid control architecture

The central design choice is to augment rather than replace the model-based controller. CTC supplies the dominant nonlinear command from the nominal robot model. The trained DDPG actor receives the tracking state and produces a bounded correction for effects that remain after nominal compensation.

External disturbance torque is applied separately to the simulated plant; it is not part of τ_cmd. This separation matters when interpreting the architecture and the torque plots: the learned action is a controller correction, while the disturbance is an independently injected plant input.

Residual DDPG design

The actor observation is a 12-dimensional vector formed from the three joint positions, three joint velocities, three position errors, and three error-rate terms. The action is a three-dimensional residual torque vector. The actor uses two 256-unit hidden layers with ReLU activations and a final tanh output that is scaled to the joint-wise bounds ±[5, 3, 2] N·m. The critic combines separate observation and action feature paths to estimate the state-action value.

Training uses off-policy DDPG with replay-buffer sampling, actor/critic updates, target networks, and exploration noise. The reward penalizes tracking error, error rate, residual magnitude, and rapid changes in the normalized residual action. Deterministic evaluation uses the saved actor with no exploration noise and no online learning or adaptation.

Constraint-aware cable-driven operation

For the modeled cable routing, the algebraic torque-to-force relationship is evaluated as τ_c = J_c(q)^T F. A meaningful pull-only solution requires nonnegative inferred cable force. In the square three-cable / three-DOF mapping, a full-rank cable Jacobian gives a unique inferred tension vector; there is no null-space redundancy available for pretension redistribution in this simplified model.

The joint-limit domain was sampled at 45 values per joint, producing 91,125 configurations. Of these, 33,921 satisfied the tested static criterion: full-rank cable Jacobian, negligible torque-reconstruction residual, and nonnegative inferred tension for nominal gravity compensation. This is 37.22% of the sampled configurations. The desired path lies inside both the sampled joint-limited workspace and the tested static nonnegative-tension subset.

Along the desired path, the cable Jacobian retains rank 3. The minimum singular value is approximately 0.01481 and the maximum two-norm condition number is approximately 71.25. The mapping therefore avoids exact rank loss on the logged path, but its least favorable postures remain sensitive: geometry or torque perturbations can be amplified in the inferred cable demand even when rank is preserved.

Simulation and model-consistency validation

The analytical equation-of-motion model, nominal CTC, trained actor, uncertainty/disturbance gates, cable mapping, and post-processing are implemented in MATLAB/Simulink R2022b. The main evaluation uses a 10 s simulation with a 0.01 s fixed step. The analytical dynamics were also compared against archived Simscape Multibody and MSC ADAMS implementations under matched nominal geometry and reference motion.

Across the archived model comparisons, the RMS discrepancy is approximately 0.98–2.67% of the corresponding analytical motion range, and the maximum absolute discrepancy is approximately 2.47–6.15%. The checks are intended to expose sign, unit, joint-order, or implementation inconsistencies; they do not establish physical or clinical model validity.

Disturbance and uncertainty evaluation

The primary evaluation separates four scenarios: C1 nominal, C2 uncertainty only, C3 disturbance only, and C4 combined uncertainty + disturbance. The uncertainty is a simultaneous +10% perturbation of selected plant parameters from 1–5 s while calibrated kinematic geometry remains fixed. Three band-limited white-noise disturbance-torque channels are active from 5–9 s. Paired controller comparisons use the same reference, nominal model, gains, initial condition, timing, and disturbance realization; the actor is not retrained between cases.

Main quantitative result: representative combined case

The detailed representative saved run is the clearest single stress-test illustration. Under the combined uncertainty-and-disturbance condition, the bounded residual controller stays closer to the desired Cartesian path and reduces the Euclidean tracking-error envelope relative to baseline CTC.

The residual controller does not obtain the tracking improvement for free. In this representative log, the maximum inferred algebraic cable demand increases from approximately 76.87 N for CTC to 86.24 N for the residual branch. This is a useful tracking–actuation-demand tradeoff, but the values remain inferred demands because the plant is not driven through a physical cable-force loop.

Robustness, case matrix, and result provenance

The research does not rely on a single saved trajectory. An independently generated four-case dataset reports RMS-error reductions of 21.1% in the nominal case, 39.4% with uncertainty only, 27.7% with disturbance only, and 41.6% in the combined case. The nominal IAE is essentially unchanged and is slightly worse for the residual controller by about 0.15%, which is why the manuscript reports multiple error metrics instead of claiming uniform improvement.

For ten disturbance seeds in the combined case, mean RMS error is 0.022945 ± 0.000533 m for CTC and 0.012916 ± 0.000082 m for CTC + residual DDPG, corresponding to a mean reduction of 43.69 ± 1.17% across those tested realizations. Interval-wise mean RMS reductions are 23.82% during the initial transient, 81.84% during the uncertainty interval, 68.45% during the disturbance interval, and 22.64% during recovery. These are empirical finite-sample robustness diagnostics, not statistical guarantees.

Residual authority and trajectory transfer

A separate action-usage audit tests whether the learned policy actually remains a correction layer. Maximum absolute residual torques are 4.86, 0.92, and 1.80 N·m for joints 1–3, corresponding to 97.2%, 30.7%, and 89.8% of their allowed bounds. Only joint 1 reaches the near-limit criterion (≥95% of its bound), and only for 0.02 s, or 0.20% of saved samples. Residual RMS torque is 11.85%, 7.79%, and 20.79% of the corresponding CTC RMS torque for the three joints.

Four nearby Cartesian initial conditions were also evaluated without retraining the actor, providing a local sensitivity check around the nominal initial state. A separate circular reference was then used as an alternate-trajectory transfer test. On that path, residual DDPG reduces RMS Cartesian error from 21.74 mm to 12.05 mm (44.56%), but the associated joint-limit channel-sample count is 44 for CTC and 101 for the residual branch.

What the results mean

The strongest interpretation is not that reinforcement learning is universally superior to model-based control. The results instead support a more specific claim: a bounded learned residual can compensate for repeatable model mismatch and disturbance while leaving the nominal nonlinear controller responsible for the main command. The largest interval-wise gains occur when the plant is deliberately moved away from the nominal model or subjected to disturbance, which is consistent with that design intent.

The cable analysis also changes how the tracking result should be read. Better Cartesian error can coincide with larger inferred cable demand, poor Jacobian conditioning, or trajectory-dependent joint-limit violations. This is why the study treats tracking, joint constraints, pull-only demand, workspace feasibility, and mapping sensitivity as complementary diagnostics rather than collapsing controller quality into a single RMS number.

Research contribution

• Hybrid control architecture: nominal nonlinear CTC is augmented by a joint-wise bounded residual DDPG policy instead of being replaced by end-to-end learned torque control.
• Non-nominal evaluation: controller comparisons cover nominal operation, simultaneous +10% plant-parameter mismatch, external disturbance, and their combination under paired conditions.
• Cable-specific feasibility analysis: Cartesian tracking is evaluated alongside joint limits, inferred pull-only cable demand, sampled static nonnegative-tension feasibility, and cable-Jacobian rank/conditioning.
• Broader quantitative evidence: RMS, peak error, IAE, ISE, disturbance-seed tests, interval-wise metrics, nearby initial states, residual-authority usage, and an alternate trajectory are used to expose both improvements and failure modes.
• Traceable interpretation: independently saved datasets are kept separate when their disturbance realizations or constraint logs differ, rather than being combined into a more convenient result.

Limitations

• The study is simulation-only and uses a planar three-link model; it does not include hardware experiments, patient interaction, human-subject testing, or clinical validation.
• No formal closed-loop stability or robust-stability proof is provided for the learned residual policy. Bounded action alone is not a safety guarantee.
• The primary evaluation uses the training trajectory; the nearby-initial-condition study is local, and the circular path is only one alternate-trajectory test.
• The circular-path result explicitly shows that improved Cartesian tracking does not imply joint-limit or constraint generalization.
• The cable-force layer is an algebraic diagnostic, not a simulated physical tension-control loop. Positive inferred tension is not equivalent to complete dynamic cable feasibility.
• No physical F_max, pretension requirement, motor/spool dynamics, cable elasticity, tension servo, or actuator-saturation model is currently defined.
• Archived training material does not retain genuine episode-by-episode learning curves or a complete checkpoint-selection rationale, so no training-curve or optimal-checkpoint claim is made.

Research directions

The next technical steps are to define physical pretension and force bounds, introduce constrained cable-force allocation with explicit infeasible-command handling, include joint-limit and cable-force margins in the learning objective, broaden randomized training and evaluation distributions, analyze practical stability/safety more formally, and move toward higher-fidelity actuator/cable models followed by hardware-oriented validation. Patient-specific adaptation and richer interaction models are longer-term directions rather than completed capabilities of the current system.

Research methods and technical skills

• nonlinear robot dynamics and computed torque control;
• residual reinforcement learning and DDPG for continuous control;
• constrained inverse kinematics and rehabilitation-oriented trajectory generation;
• cable-length Jacobian modeling and pull-only force-demand analysis;
• joint-limit, workspace, rank, singular-value, and condition-number diagnostics;
• disturbance, parametric-uncertainty, seed, interval, and initial-condition evaluation;
• MATLAB/Simulink, Simscape Multibody, and MSC ADAMS model-consistency workflows;
• quantitative tracking analysis, scientific figure generation, and provenance-aware result reporting;
• reproducible simulation and manuscript-oriented research documentation.

Research code: private working repository; a curated research-code package can be shared on request.