Diagnosis and repair of a mis-calibrated carrier-loss watchdog in a closed V.34 datapump
LiteNet Connectivity Solutions runs a modern dial-up Internet service. Its modem termination is software: the Smart Link 56K datapump, a closed 1.2 MB object file, driven over a packet-switched path rather than a copper telephone line. The service connected at 14,400 bit/s and would not go faster. The datapump advertises full V.34 capability, but attempts to negotiate it succeeded on about half of calls and returned inconsistent rates.
We identify the cause and repair it. The cause is not a channel impairment. A carrier-loss watchdog inside the datapump ends the call after one unbroken second below a signal threshold, and on this installation that threshold is 101: the out-of-range clamp of a lookup table indexed in dBm. Nobody chose the value. A path with no analog front end gives the table no index it can use, so the lookup falls through to its clamp. The criterion the watchdog then enforces, one second of quiet, is shorter than the gaps this path leaves between handshake stages, and those gaps are protocol-legal.
Static analysis of the vendor object located the defect. A deterministic replay rig confirmed it. Link-time symbol interposition corrects it, and no vendor instruction changes. In a blocked, interleaved A/B running an identical binary in both arms with a single environment variable between them, call completion rose from 19/30 to 27/30 (Fisher exact, two-sided, p = 0.030). Measured end to end, through dial, PPP negotiation, RADIUS authentication, an HTTP page retrieval and a three-minute session, the service now completes the whole path on most attempts. Every call that connects does so above 14,400 bit/s in both directions; downstream never falls below 28,800 bit/s and is 33,600 on most connects, and upstream never falls below 24,000.
Five secondary defects are also documented, four of them in the measurement apparatus rather than in the modem. One of those caused every absolute rate in earlier drafts of this work to describe a single direction of a two-direction link. Seven hypotheses advanced during the work are recorded as falsified, three of them advanced by the analysis itself and disproved by data it already held.
This work was carried out by an agentic AI system, Claude Opus 5, running in Claude Code under human direction. Appendix A sets out who did what, in detail.
LiteNet Connectivity Solutions provides dial-up Internet access. Subscribers dial a conventional telephone number; the call is carried over SIP to a server that terminates it in software and presents a PPP session, RADIUS authentication, and a proxied view of the modern web rescaled for period hardware. There is no modem bank. Every modem on the answering side is a process.
The production pool comprises thirty-two independent software modems running V.32bis at an 8,000 Hz sample rate. That configuration is stable and has been in service without modification throughout the work described here. It answers at 14,400 bit/s, which until now has been the only rate the service could advertise.
The V.34 effort predates this work. Two earlier fixes had already been established and are load-bearing for everything reported here, and are recorded so that the results below are not mistakenly attributed to the change this paper is about:
After both, the service connected on roughly half of attempts. The residual failure had a clean signature and no explanation: a log line reading vpcm: Link Error, present in 23 of 23 failures of one class and in none of 47 successes. Establishing what that line meant is the subject of this paper.
The datapump is the DSP half of slmodemd, an open host driver around a closed vendor object, dsplibs.o. The object contains complete V.32, V.32bis, V.34, V.90 and V.92 implementations, verified by symbol table inspection rather than by vendor claim. It is supplied without source.
What matters for this paper is not the datapump but what sits in front of it. Figure 1 shows the chain. A subscriber's modem drives an analog line into a terminal adapter; everything after that is packets. There is exactly one analog hop in the entire path, and it is four stages removed from the DSP.
This is the structural fact from which the defect follows. Any decision inside the datapump that compares a measured level against an absolute threshold is, on this installation, comparing two quantities that have no common reference.
The line's behaviour is not stationary. Measured over 93 consecutive calls at fixed settings, binned in twenty-minute buckets, the completion rate ran 44, 50, 75, 50, 50, 71, 57, 88, 33 and 67 per cent with no trend. A factor of 2.7 is therefore available from the line alone, and any two sequential batches can differ by that much for no reason at all.
Three rules follow, and they are the reason the results in section 5 are trustworthy where earlier results on this project were not:
The vendor object drives the datapump from a single function, vpcm_run, which is called once per fragment of 48 samples and switches on the return value of another, VPcmV34Progress. Disassembling the switch yields the complete life-or-death table:
ret 0 "Re-starting phase II", and resets the training watchdog ret 1 "Phase II completed" ret 2, 3 no change ret 4, 5 LINK ESTABLISHED — read the datapump and rates, report CONNECT ret 6, 7 still training ret 8, 9, 16 FATAL — "vpcm: Link Error" → modem_hup → NO CARRIER ret 10 "Same Line Verification Status" ret 11..15 no change
The return value is the integer at offset 4 of the datapump object. Three values are fatal, and the failure under investigation reported status 9.
Status 9 is not written by VPcmV34Progress. It is written by v34handshak, at .text+0x62a92, in a fragment that reads as follows once the stack slots are resolved (the register holding obj+4 is [esp+0x78]):
if ((short)metric < obj->lowsig_thresh) obj->lowsig_count++; else obj->lowsig_count = 0; if (obj->lowsig_count > 0x257f) *status = 9; /* hang up */
v34handshak is called once per sample from the phase-0 loop. 0x257f is 9,599, so at the 9,600 Hz DSP rate the condition is exactly one second. The counter is consecutive, so a single sample at or above the threshold resets it. Status 9 therefore means one unbroken second below a signal threshold during the handshake. It carries no information about decoding, constellation, or error rate.
The threshold is not a constant. VPcmV34SetMinimumSigLevel selects it from an eight-entry table indexed by a configured line level:
| index | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
|---|---|---|---|---|---|---|---|---|
| level (dBm) | −48 | −47 | −46 | −45 | −44 | −43 | −42 | −41 |
| threshold | 71 | 80 | 90 | 101 | 113 | 127 | 142 | 160 |
Table 1. The threshold table at .data+0xc0. The values form a clean 1 dB ladder: each is 1.122 times the last, which is 10^(1/20).
The index is computed as the configured level plus 0x30 and clamped to 3 when it falls outside 0–7. On this installation the live threshold is 101, which is the entry at index 3, which is to say the clamp. There is no dBm-calibrated level to index the table with, so the lookup has no valid answer and returns its fallback. The number was not chosen for this line; it is what the code does when it cannot choose.
Figure 2 shows the mechanism against two real calls. The handshake reaches a stage in which the far end falls quiet while the two modems turn around. That quiet is protocol-legal and, on this path, it is long: measured across 76 such windows, the far end resumes at 4.03–4.07 s, or approximately 450 ms later at 4.30–4.54 s, or after an internal restart at 8.7–9.2 s.
Half of all observed windows required longer than the watchdog allows. The distribution of arrival times is not pathological. It is simply wider than one second, which is a property of carrying V.34 over a packetised path with a terminal adapter, and which the vendor's one-second constant was never calibrated against.
The vendor object is not modified. dsplibs.o emits its call to VPcmV34Progress as an R_386_PC32 relocation against a global symbol rather than resolving it internally, which means ordinary link-time symbol resolution is sufficient to place our code in the path. One objcopy pass weakens the vendor definition and gives its body a second name; a strong definition in our own translation unit then wins the link, and calls the original through the alias.
From inside the wrapper the datapump's private state is addressable. The offsets below were derived from the disassembly and each is confirmed at runtime by reading back a value the object also prints to its own log:
| offset | field | how it was located |
|---|---|---|
| obj+0x00 | internal phase | dispatch index in VPcmV34Progress |
| obj+0x04 | status / return value | returned verbatim by VPcmV34Progress |
| obj+0x230 | low-signal threshold | written by VPcmV34SetMinimumSigLevel |
| obj+0x234 | low-signal counter | incremented in v34handshak |
| obj+0x238 | elapsed samples | zeroed by VPcmV34SetTimeOut |
| obj+0x23c | session timeout limit | seconds × 9600, same function |
| obj+0x398 | the signal metric itself | [esp+0x74]+0x134 in v34handshak |
| obj+0x4b6 | retrain threshold | logged at .text+0x676f3 |
| obj−0x0c | host train-timeout counter | vpcm_run's own state, below the object |
Table 2. Datapump structure offsets recovered by static analysis. The negative offset is not an error: vpcm_run keeps its per-call state immediately below the object it passes down, so the host's own counter is addressable from the wrapper.
The threshold at obj+0x230 is written before every fragment. Writing it once would not be sufficient: VPcmV34SetMinimumSigLevel re-arms it from the table whenever the datapump re-selects a rate, and VPcmV34InitiateRetrain calls that function, so a single write would be silently reverted by the first retrain and the setting would appear to fail on some calls and not others.
The change is bounded on both sides. The datapump's own 45-second session timeout and the host's 15.005-second acquisition budget are untouched, so a call on a genuinely dead line still terminates; what is removed is only the one-second criterion that was firing on live ones.
The intervention was evaluated in a blocked, interleaved A/B: three blocks, arms alternating, two warm-up calls discarded after each service restart, identical binary in both arms, one environment variable between them. n = 30 per arm.
Completion rose from 19/30 to 27/30. Fisher exact, two-sided, gives p = 0.030; the one-sided value, which the direction predicted from the disassembly would justify, is 0.015. We quote the two-sided figure throughout. The residual 15-second acquisition timeouts fell from 5 to 3. Mean handshake depth, an ordinal recorded on every call, including those that never complete, and therefore a more powerful statistic than completion alone, rose from 21.3 to 23.7.
Removing the one-second watchdog exposed a second constraint that had previously been masked. The host driver maintains its own acquisition budget of 3,001 fragments of 48 samples, or 15.005 seconds, and terminates the call when it expires. Before the intervention, slow calls were being killed at four seconds and the slow tail of the acquisition-time distribution therefore never existed. Afterwards it does:
| median | p75 | p90 | p95 | |
|---|---|---|---|---|
| before the intervention | 9.55 s | — | 12.10 s | — |
| after the intervention | 10.02 s | 21.48 s | 24.02 s | 33.35 s |
Table 3. Time from datapump start to link establishment, on calls that completed. The post-intervention tail is not a regression; it is the population that previously did not survive long enough to be measured.
A corresponding extension of that budget was implemented and tested. It moved every measure in the expected direction: completion 25/30 to 28/30, the timeouts it targets 5 to 2, mean depth 22.3 to 24.2, with every completion reporting 33,600 caller-side in both arms. At Fisher p = 0.424, however, the effect rests on three calls and is not established. It is not deployed, and the acceptance measurement in section 8 deliberately excludes it.
The answering modem is configured with a minimum rate of 14,400 bit/s and reports that minimum in every rate decision it logs. Across 271 decisions in one day it selected below that minimum eleven times: nine at 12,000, one at 9,600 and one at 7,200. That is 4.1% of decisions, and each becomes a working but slow session.
A floor was implemented in the bridge process, which learns the negotiated rate before it commits anything to the session: below a configured minimum it declines the call and hangs up, and the client redials. It was verified functionally rather than statistically, because a 4% event needs hundreds of calls to A/B and five to demonstrate if the rate can be forced. The caller was capped, and every resulting sub-floor connection is refused.
V.34 negotiates its two directions independently, and a modem's CONNECT result code reports its own side. The end-to-end harness parsed the CALLING modem's CONNECT and printed it in a column headed simply "rate". Every absolute rate figure produced by this work came from that column, and nothing recorded which of the two carriers it described.
The operator console had been displaying the other side for the entire study. The bridge process parses the ANSWERING modem's CONNECT into a per-session file, which the console and the content proxy both read. Over the acceptance run the harness reported 33,600 bit/s on 47 of 47 connections while the console showed 33,600, 31,200, 28,800 and 26,400 on the same calls. The discrepancy was raised by the operator from the console display; no instrument in the measurement chain could have surfaced it, because every one of them read the same side.
Neither figure was the answer. The answering modem logs both carriers on every call and always had, on a single line giving rx and tx. Its CONNECT reports the rx value, which is the upstream direction. The console was therefore showing the pessimistic half of each link. Re-measured from that log over 48 connections:
| 33,600 | 31,200 | 28,800 | 26,400 | minimum | |
|---|---|---|---|---|---|
| downstream (to subscriber) | 46 | — | 2 | — | 28,800 |
| upstream (from subscriber) | 36 | 2 | 9 | 1 | 26,400 |
Table 4. Both carriers, initial training, 48 consecutive connections. Every call cleared 14,400 bit/s in both directions. Seven mid-call retrains occurred; two moved the downstream carrier to 31,200.
Two things follow. The comparison in section 5 is unaffected: a systematically one-sided instrument applied identically to both arms still supports a difference between them, and the intervention's effect is on call completion rather than on rate. But every ABSOLUTE rate statement in the earlier drafts of this work was an overstatement of the link as a whole, and the figure a service may advertise is the floor of the slower direction, not the mode of the faster one.
Seven hypotheses were advanced during this work and subsequently falsified. They are recorded because a negative result that is not written down is re-run, and because three of them were advanced by the analysis itself and disproved by data it already possessed.
| hypothesis | how it died |
|---|---|
| The datapump's own line probe drives the failure | The probe vector is identical to three decimal places on calls that complete and calls that fail. |
| Transport loss or jitter | 22,428 packets across 21 streams, 0.0% loss on every one; mean jitter 0.09–2.19 ms. |
| The path is too quiet for the threshold, so raise the gain | Peak is −0.5 dBFS and RMS −15 to −21 dBFS. The signal is loud; added gain would clip it and nothing else. |
| The ~2 s caller silence trips the 1 s watchdog | Ten of fourteen calls whose far end returned had gaps longer than one second and completed anyway. Advanced by the analysis; falsified by its own gap measurements. |
| Calls clear a fixed 4.080 s deadline by 45 ms | Six calls in the control arm arrived after 4.080 s and survived. Advanced by the analysis; falsified by its own per-call counters. |
33,600 cannot hold, because equerr exceeds the retrain threshold | The equerr values cited were from the training phase; the threshold applies in data mode, where the value is 48–63 against a threshold of 100. Advanced by the analysis; falsified by a measurement five sections earlier in its own notes. |
| A sample above the retrain threshold triggers a retrain | 976 crossings of the threshold produced one retrain. This falsified the mechanism underlying two further experiments, both of which are consequently nulls. |
Table 5. Falsified hypotheses. The last three were the analysis's own, and each was retracted in the project record at the time rather than quietly dropped.
Four faults were found not in the modem but in the instruments used to judge it. Each had produced numbers that were believed at the time:
The controlled experiment in section 5 establishes that one field caused a large share of the failures. It does not establish what the service does. Those are different claims, and only the second one can be put in front of a customer: an A/B measures a difference between arms, and a subscriber does not experience a difference between arms.
So the intervention was deployed, the line was fixed at one configuration and asserted from the running process rather than from the service manager, and sixty consecutive calls were placed over three hours. Each call had to dial, train, bring up PPP, authenticate against RADIUS, retrieve a page over HTTP, and then hold the link for a full three minutes under a once-per-second echo. A call counted only if it did all of it.
| count | of 60 | |
|---|---|---|
| connected | 55 | 92% |
| PPP established | 55 | 92% |
| page retrieved | 55 | 92% |
| held the full three minutes | 55 | 92% |
| complete success, all four | 55 | 92% |
Table 6. Sixty consecutive calls at one configuration. The columns do not decay, which is the result: nothing that connected failed at any later stage.
Every call that connected completed the whole path. There is no attrition between connecting and succeeding: all five failures were failures to connect. That is a different service from the one measured two days earlier, where roughly half of calls connected and no instrument existed that could say whether they stayed up.
The end-to-end harness reported 33,600 bit/s on all fifty-five connections. That figure is the calling modem's, and it describes one of the two carriers (section 6.3). The figures below are read from the answering modem's own per-call log, which records both, and they include the nine mid-call retrains:
| 33,600 | 31,200 | 28,800 | 26,400 | 24,000 | floor | |
|---|---|---|---|---|---|---|
| downstream (to subscriber) | 59 | 3 | 3 | — | — | 28,800 |
| upstream (from subscriber) | 48 | 3 | 11 | 2 | 1 | 24,000 |
Table 7. All 65 rate events across the run: 55 initial trainings, 1 discarded warm-up, and 9 mid-call retrains. Every event cleared 14,400 bit/s in both directions.
The two floors differ, and the lower one only appears once retrains are counted: no call ever trained below 26,400 upstream, but one call retrained down to 24,000 and spent time there. A floor computed from connect events alone describes a moment rather than a session, and would have been quoted at 26,400.
Five calls did not connect: one early, one in the middle, and three consecutively near the end. The cluster is worth more attention than the count. This line's connect rate is known to vary between 33% and 88% within a single hour at fixed settings (section 9.1), and three consecutive failures are what that variation looks like from inside a single run. Sixty calls over three hours cannot distinguish that from the onset of a new fault, and no claim is made either way.
Nine mid-call retrains occurred across the fifty-five held calls, concentrated in two of them; most calls carried none. A retrain costs roughly ten seconds of carrier and is invisible to a subscriber beyond a stall, which is why the harness had to be taught to distinguish one from a dropped call before any of these numbers could be trusted (section 7.1).
The single largest threat is the line itself. Over 93 consecutive calls at fixed settings, binned in twenty-minute buckets, completion ran 44, 50, 75, 50, 50, 71, 57, 88, 33 and 67 per cent with no trend. A factor of 2.7 is available from the channel alone.
What that permits and forbids, precisely:
Every result was measured against a single answering installation and, with one exception, a single calling modem: a Conexant-based USB unit. A hardware control was run in which two physical modems called each other across the same adapter and PBX with no LiteNet software in the path at all; it completed 4 of 10 calls and topped out at 24,000 bit/s, which is statistically indistinguishable from the software path's contemporaneous rate (Fisher p = 1.000). That control establishes that the path, not the software, imposes a substantial share of the difficulty. It does not establish that the watchdog behaves identically against every caller chipset.
The intervention removes one specific termination criterion. It does not demonstrate that the criterion is wrong in the vendor's intended deployment, on a real telephone line with a calibrated front end, one unbroken second below −45 dBm is a reasonable definition of a dead carrier. The claim made here is narrower and, we think, unimpeachable: the threshold reaching this installation is the table's out-of-range clamp rather than a chosen value, and applying it costs a quarter of all calls.
Everything needed to repeat this is published alongside the paper. The vendor object is deliberately not among it: the build ships the objcopy invocation, and the reader runs it against their own copy from their own sl-modem-daemon installation.
# 1. build a vendor tree with the wrapper interposed (asserts the call site binds) artifacts/wrapper/apply_v34_progress_hook.sh <base-vendor-dir> <new-vendor-dir> # 2. run one call with the wrapper only observing, and read the threshold it finds LITENET_V34_PROG=1 ... # logs: 'lowsig threshold 101, timeout 432000 samples' # 3. run one call with the criterion disabled LITENET_V34_LOWSIG=-1000000
A single call is enough to confirm the threshold is 101 and that it is being applied. Confirming what it costs requires the A/B.
Each figure and table in this paper is produced by a script in the artifact repository under artifacts/analysis/, reading the sanitized per-call logs in artifacts/data/. Any number quoted here that cannot be regenerated from those inputs has been removed rather than left standing.
The closest cousin to this system is D-Modem, an open-source SIP softmodem published by Stroz Friedberg. It shares the architecture almost exactly: the same slmodemd host driver, the same closed dsplibs.o datapump, and the same substitution of packets for an analog front end. It is licensed GPL-2.0.
Its README documents the same symptom this paper diagnoses, and documents it as an unexplained limitation. Verbatim:
That is an independent installation, built by different people for a different purpose, arriving at the same ceiling and the same workaround, which is to pin the modulation low by hand, without an account of the cause. Section 3 supplies the account, and it is not specific to LiteNet: any deployment that drives this datapump over a packet path inherits a carrier-loss threshold selected by a table lookup that cannot succeed. We have staged an issue and a patch for that project.
More broadly, the situation belongs to a familiar class: software written against an assumed hardware environment, run in an environment that satisfies its interfaces but not its assumptions. The interposition technique used here, weakening a symbol in a relocatable object so that ordinary link-time resolution admits an observer, is long-standing practice; what is perhaps less common is using it as a diagnostic instrument on a datapump with no source, and asserting at build time that the interposition actually took effect.
A one-second timer in a closed vendor object held this service at 14,400 bit/s. The timer compares a signal level against a threshold nobody chose: 101, which is what an eight-entry table indexed in dBm returns when it cannot be indexed. Samples arriving over packets carry no dBm reference, so it never can be. One unbroken second of quiet is shorter than the gaps this path leaves between handshake stages.
Correcting it changes none of the vendor's code. The object publishes the relocation that makes link-time interposition sufficient, and the fix writes one field before each fragment.
State the benefit exactly: the change does not make the modem faster. Every call that completed in either arm of the controlled experiment reported 33,600 bit/s, the V.34 maximum, at the calling modem, with the change and without it. That reading covers one direction (section 6.3); it was taken identically in both arms, so the comparison is unaffected. The change stops the modem ending calls that were working. The speed was there all along and the calls were being hung up before they could use it.
Two lessons outlast this particular bug. A value printed in a log is not evidence that the code uses it the way its name suggests: three parameters in this object are accepted, reported, and then ignored in the documented sense. And on a channel that drifts by a factor of 2.7 within the hour, an instrument is a hypothesis. Three of the defects we found were in the measurement apparatus, and every one had already produced numbers somebody believed.
The technical work reported in this paper, meaning the reverse engineering of the vendor datapump, the identification of the watchdog and its mis-calibration, the derivation of the structure offsets, the design and construction of the interposition mechanism, the measurement instruments, the experimental design, the statistical analysis, and the successive falsification of incorrect hypotheses including several of its own, was performed by Claude Code, an agentic software engineering system produced by Anthropic, operating under the direction of Alexander Szeremeta.
Mr Szeremeta's contribution was direction. He specified the objective, insisted on it when intermediate results suggested it was unreachable, supplied and physically administered the hardware, granted the access under which the work was carried out, and set the standard of evidence, requiring that a result be reliable and reproducible rather than a favourable sample. He did not perform the disassembly, write the analysis, design the experiments, or author the code. He has asked that this be stated plainly rather than softened, and it is stated plainly: he steered, and the engineering was done by the machine.
Specifics, so that the claim is checkable rather than rhetorical. The disassembly of vpcm_run, VPcmV34Progress and v34handshak; the recovery of the nine structure offsets in Table 2 and their runtime confirmation; the objcopy interposition and the build-time assertion that it binds; four measurement instruments written specifically because the existing ones could not answer the question at hand: an arrival-time analyser, a hold-time A/B harness, a line-state setter that asserts from /proc, and a per-gap logger in the bridge; and seven falsified hypotheses, of which three were generated by the system itself and retracted by it in the project record at the time, not quietly dropped.
No text in this paper was written by a person other than as stated above. The byline records the person who directed and owns the work. Generative AI is not listed as an author, in line with ACM policy, and this disclosure appears as a marked section in the body of the paper, in line with USENIX policy.
slmodemd and the dsplibs.o datapump), 2001–2005. The tree carries a single licence file, a 3-clause BSD licence, which permits use and modification in source and binary form and contains no restriction on reverse engineering. The DSP component is supplied as a binary-only object; Debian distributes the package in non-free.