]> git.scottworley.com Git - tattlekey/log
tattlekey
13 months agoDescribe this project
Scott Worley [Wed, 11 Oct 2023 02:01:25 +0000 (19:01 -0700)]
Describe this project

13 months agoserver: Replay log
Scott Worley [Wed, 11 Oct 2023 01:41:36 +0000 (18:41 -0700)]
server: Replay log

13 months agoserver: Denormalized Message
Scott Worley [Wed, 11 Oct 2023 01:36:27 +0000 (18:36 -0700)]
server: Denormalized Message

Oh, wow, implementing deserialization in serde by hand is painful.
This hurts less.

13 months agoserver: Hand-implement Serialize
Scott Worley [Wed, 11 Oct 2023 01:16:08 +0000 (18:16 -0700)]
server: Hand-implement Serialize

Mostly for symmetry with Deserialize, for which the lack of #[serde(flatten)]
support hurts much wors.

13 months agoserver: SystemTime → u64 of seconds since UNIX_EPOCH
Scott Worley [Wed, 11 Oct 2023 00:43:31 +0000 (17:43 -0700)]
server: SystemTime → u64 of seconds since UNIX_EPOCH

13 months agoserver: Extract merge_message()
Scott Worley [Wed, 11 Oct 2023 00:08:19 +0000 (17:08 -0700)]
server: Extract merge_message()

13 months agoserver: Bind port before messing with log files as a lock
Scott Worley [Tue, 10 Oct 2023 23:57:22 +0000 (16:57 -0700)]
server: Bind port before messing with log files as a lock

Rather than messing with flock, fnctl-F_SETLK, etc., just bind the port
before touching the log file & let the failure to bind the port keep
two instances of the server from stepping on each others' log i/o.

13 months agoserver: extract open_log_for_writing()
Scott Worley [Tue, 10 Oct 2023 23:56:45 +0000 (16:56 -0700)]
server: extract open_log_for_writing()

13 months agoserver: Only write CSV headers when initially creating the log file
Scott Worley [Tue, 10 Oct 2023 23:51:57 +0000 (16:51 -0700)]
server: Only write CSV headers when initially creating the log file

13 months agoserver: Log file name constant
Scott Worley [Tue, 10 Oct 2023 23:45:39 +0000 (16:45 -0700)]
server: Log file name constant

13 months agoserver: Write CSV to log file
Scott Worley [Tue, 10 Oct 2023 23:44:08 +0000 (16:44 -0700)]
server: Write CSV to log file

13 months agoserver: Write reports as CSV
Scott Worley [Tue, 10 Oct 2023 23:34:12 +0000 (16:34 -0700)]
server: Write reports as CSV

13 months agoserver: Drop `ago` field
Scott Worley [Tue, 10 Oct 2023 23:16:20 +0000 (16:16 -0700)]
server: Drop `ago` field

13 months agoserver: Merge keystroke time ranges
Scott Worley [Tue, 10 Oct 2023 23:12:31 +0000 (16:12 -0700)]
server: Merge keystroke time ranges

13 months agoserver: Combine ago & clock to get original event time
Scott Worley [Tue, 10 Oct 2023 22:45:38 +0000 (15:45 -0700)]
server: Combine ago & clock to get original event time

13 months agoserver: Parse messages
Scott Worley [Tue, 10 Oct 2023 22:34:58 +0000 (15:34 -0700)]
server: Parse messages

13 months agoclient: New epoch if seq wraps
Scott Worley [Tue, 10 Oct 2023 21:48:33 +0000 (14:48 -0700)]
client: New epoch if seq wraps

13 months agohousing: Clarify license
Scott Worley [Tue, 10 Oct 2023 21:39:51 +0000 (14:39 -0700)]
housing: Clarify license

13 months agoserver: Clarify license
Scott Worley [Tue, 10 Oct 2023 21:27:03 +0000 (14:27 -0700)]
server: Clarify license

13 months agoserver: Receive packets
Scott Worley [Tue, 10 Oct 2023 17:20:43 +0000 (10:20 -0700)]
server: Receive packets

13 months agoserver: Start
Scott Worley [Tue, 10 Oct 2023 16:55:50 +0000 (09:55 -0700)]
server: Start

13 months agoclient: Change wire format!: Add epoch
Scott Worley [Tue, 10 Oct 2023 09:40:57 +0000 (02:40 -0700)]
client: Change wire format!: Add epoch

This helps the server de-dupe: Records with the same sequence number but
different epochs do not describe the same event.

13 months agoclient: Change wire format!: Expand `ago` field: 16 bits → 32 bits
Scott Worley [Tue, 10 Oct 2023 09:27:57 +0000 (02:27 -0700)]
client: Change wire format!: Expand `ago` field: 16 bits → 32 bits

16-bit values of seconds only go up to 18 hours, and sometimes a little
more than that would be nice.

32 is overkill.  24 would have been fine, but would have been slightly
annoying to encode and parse.

13 months agoclient: Bump default config_resend_count 5 → 16
Scott Worley [Tue, 10 Oct 2023 09:09:44 +0000 (02:09 -0700)]
client: Bump default config_resend_count 5 → 16

13 months agoclient: Explain the exponential re-send delay
Scott Worley [Tue, 10 Oct 2023 09:09:19 +0000 (02:09 -0700)]
client: Explain the exponential re-send delay

13 months agoclient: Specify license
Scott Worley [Tue, 10 Oct 2023 08:40:21 +0000 (01:40 -0700)]
client: Specify license

13 months agoclient: lint nits
Scott Worley [Tue, 10 Oct 2023 08:34:01 +0000 (01:34 -0700)]
client: lint nits

13 months agoclient: const config
Scott Worley [Tue, 10 Oct 2023 08:27:46 +0000 (01:27 -0700)]
client: const config

13 months agoclient: Make the queue size bounds configurable
Scott Worley [Tue, 10 Oct 2023 08:14:15 +0000 (01:14 -0700)]
client: Make the queue size bounds configurable

13 months agoclient: Hold presses in queues, one queue per send_count
Scott Worley [Tue, 10 Oct 2023 07:50:42 +0000 (00:50 -0700)]
client: Hold presses in queues, one queue per send_count

This lets us efficiently hold many more presses with pending resends
and eliminates the failure mode in which young presses could be dropped
because the sleeps_heap was full of old presses.

Instead of storing each press in the sleeps_heap, we now only store up
to config_resend_count entries in the sleeps_heap.

Old presses now only compete with similarly-old presses for retention
space.

13 months agoclient: xcalloc
Scott Worley [Tue, 10 Oct 2023 07:12:39 +0000 (00:12 -0700)]
client: xcalloc

13 months agoclient: get_press_due_for_resend()
Scott Worley [Tue, 10 Oct 2023 06:05:13 +0000 (23:05 -0700)]
client: get_press_due_for_resend()

This removes the last references to pp's internals outside press.c.

13 months agoclient: create_press() → add_press()
Scott Worley [Tue, 10 Oct 2023 06:04:17 +0000 (23:04 -0700)]
client: create_press() → add_press()

13 months agoclient: Bundle press_t[] & pheap_t* together as press_pile_t
Scott Worley [Tue, 10 Oct 2023 05:21:36 +0000 (22:21 -0700)]
client: Bundle press_t[] & pheap_t* together as press_pile_t

13 months agoclient: Rename sends.c → press.c
Scott Worley [Tue, 10 Oct 2023 04:53:03 +0000 (21:53 -0700)]
client: Rename sends.c → press.c

13 months agoclient: Rename: send → press
Scott Worley [Tue, 10 Oct 2023 04:52:14 +0000 (21:52 -0700)]
client: Rename: send → press

13 months agoclient: Rename: set_send_alarm → set_resend_alarm
Scott Worley [Tue, 10 Oct 2023 04:42:11 +0000 (21:42 -0700)]
client: Rename: set_send_alarm → set_resend_alarm

13 months agoclient: Rename the event enum values
Scott Worley [Tue, 10 Oct 2023 04:39:06 +0000 (21:39 -0700)]
client: Rename the event enum values

13 months agoclient: Rename: send_report -> send_report_packet
Scott Worley [Tue, 10 Oct 2023 04:33:44 +0000 (21:33 -0700)]
client: Rename: send_report -> send_report_packet

13 months agoclient: Extract set_send_alarm()
Scott Worley [Tue, 10 Oct 2023 04:29:10 +0000 (21:29 -0700)]
client: Extract set_send_alarm()

13 months agoclient: next_scheduled_send()
Scott Worley [Tue, 10 Oct 2023 02:46:28 +0000 (19:46 -0700)]
client: next_scheduled_send()

13 months agoclient: create_send()
Scott Worley [Tue, 10 Oct 2023 00:10:27 +0000 (17:10 -0700)]
client: create_send()

13 months agoclient: Start moving pending-sends logic out to sends.c
Scott Worley [Mon, 9 Oct 2023 23:51:08 +0000 (16:51 -0700)]
client: Start moving pending-sends logic out to sends.c

13 months agoclient: Exponential backoff resend intervals
Scott Worley [Mon, 9 Oct 2023 22:17:04 +0000 (15:17 -0700)]
client: Exponential backoff resend intervals

13 months agoclient: Stop sleeping separately for each re-send
Scott Worley [Mon, 9 Oct 2023 21:53:25 +0000 (14:53 -0700)]
client: Stop sleeping separately for each re-send

Instead, determine when the next send, in the whole pile of sends,
needs to happen and sleep until then.

This is the first functionality in this project that is more than simple
glue code.  This is the first non-trivial functionality that has some
internal logic to it.

I'd really like to write a test for it.

This environment (C, CMake, pico-sdk) makes testing hard!  :(

I'd like to build the test as a host/native executable and run it on the
build machine at build time, even though the primary/production use of
the code under test would be on the pico, cross-compiled.

I know testing this way is imperfect: It would fail to catch the entire
class of bugs that are caused by the code under test making architecture
assumptions that are valid on the build host but invalid on the cross
target.  But it would still be very useful for all other classes of bugs!

This usage is very much not supported:

1. CMake doesn't support it at all: One toolchain per language:
   https://discourse.cmake.org/t/compile-unit-test-natively-and-cross-compile-code-for-embedded-target/3607
   https://discourse.cmake.org/t/using-multiple-toolchains-in-the-same-project/1505

2. Even if I'm confident in my ability to write portable C that runs
   correctly on both build-host and cross-target architectures, I can't
   rely on pico-sdk's pheap library to be portable in this way.  It is
   super-not-portable-at-all; it will never run on the build host.

3. If I use a different, portable heap library, I'm duplicating code
   already in the pico-sdk standard library.  The pico is small enough
   already.  I'd rather not deploy two heap libraries.

4. I could define an interface to a heap library & use the pico's heap
   library when deploying to the pico & some other heap library when testing
   on the native machine, but

   a. My interface-to-pico's-heap-library code goes untested.
   b. The interface-to-a-heap-library code would likely add overhead.
   c. That's a bunch of extra code to write.  :(

5. Even if I worked through the interfacing-with-pico's-heap library issue
   and made this a separate, independent, portable library, I'd still
   be left with the problem of composing the two pieces.  Do I build
   a library .so separately, spinning up a separate Debian VM for that?
   Does pico-sdk even support building libraries?  Or do I copy the source
   of the library into the executable's source tree & make building the
   library part of the executable's build process?  Ick.  :(

6. Other option: Build a test intended to run on the pico.  It's nuts
   that this is the least-nuts option.  :(

I am very discouraged.  :(

I don't even bother to keep this functionality separate with a clean
interface for the test harness.  :(

So, no tests.  :~(

For now.

Advice welcome.

13 months agoclient: Extract pending-send object as send_t
Scott Worley [Mon, 9 Oct 2023 20:35:11 +0000 (13:35 -0700)]
client: Extract pending-send object as send_t

13 months agoclient: Move queue servicing to service_queue()
Scott Worley [Mon, 9 Oct 2023 19:51:43 +0000 (12:51 -0700)]
client: Move queue servicing to service_queue()

13 months agoclient: Interleave accepting new button presses & re-sends
Scott Worley [Mon, 9 Oct 2023 19:35:15 +0000 (12:35 -0700)]
client: Interleave accepting new button presses & re-sends

13 months agoclient: queue_try_add_ignoring_errors()
Scott Worley [Mon, 9 Oct 2023 19:34:29 +0000 (12:34 -0700)]
client: queue_try_add_ignoring_errors()

13 months agoclient: Set up for multiple types of events to go into the queue
Scott Worley [Mon, 9 Oct 2023 18:56:40 +0000 (11:56 -0700)]
client: Set up for multiple types of events to go into the queue

13 months agoclient: Prefix all the config settings with "config_"
Scott Worley [Mon, 9 Oct 2023 18:32:22 +0000 (11:32 -0700)]
client: Prefix all the config settings with "config_"

13 months agoclient: Calculate 'ago' with a clock
Scott Worley [Mon, 9 Oct 2023 08:02:32 +0000 (01:02 -0700)]
client: Calculate 'ago' with a clock

13 months agoclient: Work in 1-second granularity
Scott Worley [Mon, 9 Oct 2023 07:49:54 +0000 (00:49 -0700)]
client: Work in 1-second granularity

13 months agoclient: Send each report multiple times
Scott Worley [Mon, 9 Oct 2023 03:49:43 +0000 (20:49 -0700)]
client: Send each report multiple times

UDP is unreliable.

It would be better to re-send over a much longer interval, interleaving
re-sends from different presses.  Do this simple re-sending for now.

13 months agoclient: Less chatty with the LED
Scott Worley [Mon, 9 Oct 2023 03:42:35 +0000 (20:42 -0700)]
client: Less chatty with the LED

13 months agoclient: debounce
Scott Worley [Mon, 9 Oct 2023 02:15:49 +0000 (19:15 -0700)]
client: debounce

13 months agoclient: Initial wifi connection time: 10s → 90s
Scott Worley [Sun, 8 Oct 2023 07:14:57 +0000 (00:14 -0700)]
client: Initial wifi connection time: 10s → 90s

No reason to give up so aggressively.

Be robust against the AP restarting, which takes ~1 minute.

13 months agoclient: Don't sleep in interrupt context
Scott Worley [Sun, 8 Oct 2023 07:13:36 +0000 (00:13 -0700)]
client: Don't sleep in interrupt context

13 months agoclient: Respond to button press
Scott Worley [Sun, 8 Oct 2023 06:29:49 +0000 (23:29 -0700)]
client: Respond to button press

13 months agoclient: Explain config settings
Scott Worley [Sun, 8 Oct 2023 04:45:13 +0000 (21:45 -0700)]
client: Explain config settings

13 months agoclient: net: Fix port byte order
Scott Worley [Sun, 8 Oct 2023 04:34:50 +0000 (21:34 -0700)]
client: net: Fix port byte order

Docs ( https://www.nongnu.org/lwip/2_0_x/group__udp__raw.html ) say
"ipaddr & port are expected to be in the same byte order as in the pcb,"
which apparently means _don't_ run it through htons().

13 months agoclient: Send UDP packets
Scott Worley [Sun, 8 Oct 2023 04:08:58 +0000 (21:08 -0700)]
client: Send UDP packets

13 months agoclient: Move config to config.c
Scott Worley [Sun, 8 Oct 2023 01:18:39 +0000 (18:18 -0700)]
client: Move config to config.c

13 months agohousing: Extra length for cord grip
Scott Worley [Sat, 7 Oct 2023 18:28:42 +0000 (11:28 -0700)]
housing: Extra length for cord grip

13 months agohousing: Grip board tighter
Scott Worley [Sat, 7 Oct 2023 18:16:00 +0000 (11:16 -0700)]
housing: Grip board tighter

13 months agohousing: Flatten
Scott Worley [Sat, 7 Oct 2023 18:11:57 +0000 (11:11 -0700)]
housing: Flatten

13 months agohousing: Adjust board fit
Scott Worley [Sat, 7 Oct 2023 15:33:13 +0000 (08:33 -0700)]
housing: Adjust board fit

13 months agohousing: Basic shape
Scott Worley [Sat, 7 Oct 2023 09:08:54 +0000 (02:08 -0700)]
housing: Basic shape

Not solved yet: Cord grip/anchor

13 months agohousing: Start
Scott Worley [Sat, 7 Oct 2023 08:18:21 +0000 (01:18 -0700)]
housing: Start

13 months agoclient: Move blink stuff to separate file
Scott Worley [Sat, 30 Sep 2023 01:48:53 +0000 (18:48 -0700)]
client: Move blink stuff to separate file

13 months agoclient: Connect to wifi
Scott Worley [Fri, 29 Sep 2023 21:56:48 +0000 (14:56 -0700)]
client: Connect to wifi

13 months agoclient: Fancier blinking
Scott Worley [Fri, 29 Sep 2023 21:48:28 +0000 (14:48 -0700)]
client: Fancier blinking

13 months agoclient: Build wifi deps
Scott Worley [Fri, 29 Sep 2023 21:40:01 +0000 (14:40 -0700)]
client: Build wifi deps

13 months agoclient: Get lwipopts.h from pico-examples
Scott Worley [Fri, 29 Sep 2023 21:39:32 +0000 (14:39 -0700)]
client: Get lwipopts.h from pico-examples

13 months agoclient: Build in a Debian VM
Scott Worley [Fri, 29 Sep 2023 19:37:21 +0000 (12:37 -0700)]
client: Build in a Debian VM

This finally makes it work.  Hurray!

Other things I tried that didn't help:
  * Other versions of cross gcc (8, 9, 10, 11 12)
  * Other versions of host gcc (10)
  * Other versions of pico-sdk (1.3.1, 1.4.0)

arm-none-eabi-gcc as packaged in Debian works.
arm-none-eabi-gcc as packaged in Nix doesn't.
I haven't yet looked further into why.

13 months agoclient: Use standard <nixpkgs>
Scott Worley [Fri, 29 Sep 2023 19:31:25 +0000 (12:31 -0700)]
client: Use standard <nixpkgs>

Do the "fetchSubmodules = true;" override here

13 months agoclient: Install the .uf2
Scott Worley [Fri, 29 Sep 2023 19:21:25 +0000 (12:21 -0700)]
client: Install the .uf2

13 months agoclient: Switch to pico_w blink example
Scott Worley [Fri, 29 Sep 2023 19:20:53 +0000 (12:20 -0700)]
client: Switch to pico_w blink example

13 months agoclient: Start with the 'blink' example
Scott Worley [Thu, 28 Sep 2023 00:46:26 +0000 (17:46 -0700)]
client: Start with the 'blink' example