Auth & token lifecycle

LatticePulse polls X and Reddit every few minutes. Both authenticate with OAuth 2.0, and both issue access tokens measured in hours. A process that runs unattended for months will therefore refresh its credentials thousands of times, and every one of those refreshes is an opportunity to stop working forever.

The dangerous part is what failure looks like. If the refresh token is revoked — a user changes a password, an app’s permissions are reviewed, a provider rotates something — ingestion from that source simply returns nothing. The heatmap keeps rendering. The scores keep computing. A source producing zero events is indistinguishable from a source where nothing is happening, and “nothing is happening” is a perfectly plausible reading of a quiet Sunday.

By the time someone notices that X has contributed no signals since Tuesday, the baseline has been polluted: every ticker’s attention velocity has been computed against a window that was missing a whole source. The scores aren’t just stale, they’re wrong in a way that survives the fix.

CryptoFundingWatch has the mirror-image problem. Exchange APIs authenticate with signed API keys rather than OAuth, so nothing expires on a timer — but nothing refreshes either. A rotated or rate-limit-suspended key fails the same silent way, and because there is no expiry event to hook into, there is nothing to monitor except the absence of data. The lesson transfers: the credential needs a heartbeat, not just a refresh.

SEC EDGAR, by contrast, needs no credential at all — just a declared User-Agent. Worth noting because it is the only one of the four sources that cannot fail this way, and it is not a coincidence that it is also the least fragile.

Why PKCE. Without it the authorization code is a bearer token travelling through a redirect — a URL, which lands in browser history, server logs and Referer headers. Anyone who captures it can exchange it for real tokens, because the token endpoint asks only for the code and a client ID.

PKCE makes the code useless on its own. The client generates a random verifier, sends only its hash as a challenge, and must present the original verifier at exchange time. A stolen code without the verifier exchanges for nothing. Ten lines, specified in RFC 7636.

The refresh logic is not the hard part; the monitoring is. Going in, this bench was about getting PKCE and rotation right. Those took an afternoon and every library does them already. The finding that actually changed how both systems are built is that a credential failure produces no error anywhere — it produces an absence, and absences don’t page anyone. Every source now carries a last-successful-fetch timestamp and alerts on staleness, which catches key rotation, revocation, rate-limit suspension and a silently changed endpoint with one mechanism.

Refreshing reactively creates a problem that didn’t need to exist. Waiting for a 401 puts the expiry inside a live request, which then has to be retried, which means the request has to be safe to repeat. Refreshing five minutes early removes the entire class of problem for the cost of one extra call per token lifetime.

Distinct failure branches are worth the extra code. The bench handles cancelled consent, state mismatch, replayed code and revoked grant as four separate outcomes rather than one 500. Only one of them — invalid_grant on refresh — is terminal and needs a human. Collapsing them means either paging someone for a user who clicked cancel, or retrying forever on something that will never succeed.

Not everything needs a credential, and that is an architectural option. SEC EDGAR is the least fragile of LatticePulse’s four sources purely because it has nothing to revoke. Where a public or key-free endpoint gives the same data, it is worth a meaningful loss of convenience.