Replace use of "marker" with "log field"

"Marker" is an SLF4J-specific term, that was not originally designed for
adding key-value data to logs in the way that we use it. Markers are
more like "tags" on a log. But the logstash-logback-encoder library,
which we use to attach structured key-value data to logs, use markers
to pass this data.

Initially, I thought that using the term "marker" made sense, since that
is the underlying concept we use from logstash-logback-encoder. But this
confuses the logstash-specific key-value marker concept with the more
general SLF4j concept of a marker. In addition, no other languages that
I know of use the term "marker" to describe key-value data added to
logs.

So with this in mind, I've decided to instead use the term "log field".
After all, the way we use markers internally in the library is just to
add key-value pairs to the log output - i.e., fields.

Author: hermannm

README.md

@@ -1,7 +1,6 @@
 # devlog-kotlin
 
-Logging library for Kotlin JVM, that thinly wraps SLF4J and Logback to provide a more ergonomic API,
-and to use `kotlinx.serialization` for log marker serialization instead of Jackson.
+Logging library for Kotlin JVM, that thinly wraps SLF4J and Logback to provide a more ergonomic API.
 
 Published on Maven Central: https://central.sonatype.com/artifact/dev.hermannm/devlog-kotlin
 
@@ -38,7 +37,7 @@ fun example() {
 }
 ```
 
-You can also add _log markers_ (structured key-value data) to your logs. The `addMarker` method uses
+You can also add _log fields_ (structured key-value data) to your logs. The `addField` method uses
 `kotlinx.serialization` to serialize the value.
 
 ```kotlin
@@ -50,27 +49,36 @@ fun example() {
   val user = User(id = 1, name = "John Doe")
 
   log.info {
-    addMarker("user", user)
+    addField("user", user)
     "Registered new user"
   }
 }
 ```
 
-This will give the following log output (if outputting logs as JSON with
-`logstash-logback-encoder`):
+When outputting logs as JSON (using [`logstash-logback-encoder`](#setting-up-with-logback)), the
+key/value given to `addField` is added to the logged JSON object (see below). This allows you to
+filter and query on the field in the log analysis tool of your choice, in a more structured manner
+than if you were to just use string concatenation.
 
 ```jsonc
-{ "message": "Registered new user", "user": { "id": 1, "name": "John Doe" } }
+{
+  "message": "Registered new user",
+  "user": {
+    "id": 1,
+    "name": "John Doe"
+  }
+  // ...timestamp etc.
+}
 ```
 
-If you want to add markers to all logs within a scope, you can use `withLoggingContext`:
+If you want to add fields to all logs within a scope, you can use `withLoggingContext`:
 
 ```kotlin
-import dev.hermannm.devlog.marker
+import dev.hermannm.devlog.field
 import dev.hermannm.devlog.withLoggingContext
 
 fun processEvent(event: Event) {
-  withLoggingContext(marker("event", event)) {
+  withLoggingContext(field("event", event)) {
     log.debug { "Started processing event" }
     // ...
     log.debug { "Finished processing event" }
@@ -85,21 +93,20 @@ fun processEvent(event: Event) {
 { "message": "Finished processing event", "event": { /* ... */ } }
 ```
 
-Note that `withLoggingContext` uses a thread-local to provide markers to the scope, so it won't work
-with Kotlin coroutines and `suspend` functions (though it does work with Java virtual threads). An
-alternative that supports coroutines may be added in a future version of the library.
+Note that `withLoggingContext` uses a thread-local to provide log fields to the scope, so it won't
+work with Kotlin coroutines and `suspend` functions (though it does work with Java virtual threads).
+An alternative that supports coroutines may be added in a future version of the library.
 
 Finally, you can attach a `cause` exception to logs:
 
 ```kotlin
-fun example(user: User) {
+fun example( {
   try {
-    storeUser(user)
+    callExternalService()
   } catch (e: Exception) {
     log.error {
       cause = e
-      addMarker("user", user)
-      "Failed to store user in database"
+      "Request to external service failed"
     }
   }
 }
@@ -130,9 +137,10 @@ See the [Usage docs](https://github.com/logfellow/logstash-logback-encoder#usage
 ## Implementation
 
 All the methods on `Logger` are `inline`, and don't do anything if the log level is disabled - so
-you only pay for marker serialization and log message concatenation if it's actually logged.
+you only pay for log field serialization and message concatenation if it's actually logged. Inlining
+the logger methods avoids having to allocate a function object for the lambda argument.
 
-Elsewhere in the library, we use inline value classes to wrap Logback APIs, to get as close as
+Elsewhere in the library, we use inline value classes when wrapping Logback APIs, to get as close as
 possible to a zero-cost abstraction.
 
 ## Credits

src/main/kotlin/dev/hermannm/devlog/ExceptionWithLogMarkers.kt renamed to src/main/kotlin/dev/hermannm/devlog/ExceptionWithLogFields.kt

@@ -1,25 +1,25 @@
 package dev.hermannm.devlog
 
 /**
- * Interface to allow you to attach [log markers][LogMarker] to exceptions. When passing a `cause`
+ * Interface to allow you to attach [log fields][LogField] to exceptions. When passing a `cause`
  * exception to one of the methods on [Logger], it will check if the given exception implements this
- * interface, and if it does, these log markers will be attached to the log.
+ * interface, and if it does, these fields will be added to the log.
  *
  * This is useful when you are throwing an exception from somewhere down in the stack, but do
- * logging further up the stack, and you have structured data that you want to attach to the logged
- * exception. In this case, one may typically resort to string concatenation, but this interface
+ * logging further up the stack, and you have structured data that you want to attach to the
+ * exception log. In this case, one may typically resort to string concatenation, but this interface
  * allows you to have the benefits of structured logging for exceptions as well.
  *
  * ### Example
  *
  * ```
  * import dev.hermannm.devlog.Logger
- * import dev.hermannm.devlog.WithLogMarkers
- * import dev.hermannm.devlog.marker
+ * import dev.hermannm.devlog.WithLogFields
+ * import dev.hermannm.devlog.field
  *
- * class InvalidUserData(user: User) : RuntimeException(), WithLogMarkers {
+ * class InvalidUserData(user: User) : RuntimeException(), WithLogFields {
  *   override val message = "Invalid user data"
- *   override val logMarkers = listOf(marker("user", user))
+ *   override val logFields = listOf(field("user", user))
  * }
  *
  * fun storeUser(user: User) {
@@ -43,7 +43,7 @@ package dev.hermannm.devlog
  * ```
  *
  * The `log.error` would then give the following log output (using `logstash-logback-encoder`), with
- * the `user` log marker from `InvalidUserData` attached:
+ * the `user` log field from `InvalidUserData` attached:
  * ```
  * {
  *   "message": "Failed to store user",
@@ -56,18 +56,18 @@ package dev.hermannm.devlog
  * }
  * ```
  */
-interface WithLogMarkers {
-  /** Will be attached to a log if this is passed through a `cause` parameter to [Logger]. */
-  val logMarkers: List<LogMarker>
+interface WithLogFields {
+  /** Will be attached to the log when passed through `cause` to one of [Logger]'s methods. */
+  val logFields: List<LogField>
 }
 
 /**
- * Base exception class implementing the [WithLogMarkers] interface for attaching structured data to
+ * Base exception class implementing the [WithLogFields] interface for attaching structured data to
  * the exception when it's logged. If you don't want to create a custom exception and implement
- * [WithLogMarkers] on it, you can use this class instead.
+ * [WithLogFields] on it, you can use this class instead.
  */
-open class ExceptionWithLogMarkers(
+open class ExceptionWithLogFields(
     override val message: String?,
-    override val logMarkers: List<LogMarker>,
+    override val logFields: List<LogField>,
     override val cause: Throwable? = null,
-) : RuntimeException(), WithLogMarkers
+) : RuntimeException(), WithLogFields

src/main/kotlin/dev/hermannm/devlog/LogBuilder.kt

@@ -5,7 +5,30 @@ import ch.qos.logback.classic.spi.ThrowableProxy
 import kotlinx.serialization.SerializationStrategy
 import net.logstash.logback.marker.SingleFieldAppendingMarker
 
-/** Class used in the logging methods on [Logger] to add markers/cause exception to logs. */
+/**
+ * Class used in the logging methods on [Logger], allowing you to set a [cause] exception and
+ * [add structured key-value data][addField] to a log.
+ *
+ * ### Example
+ *
+ * ```
+ * private val log = Logger {}
+ *
+ * fun example(user: User) {
+ *   try {
+ *     storeUser(user)
+ *   } catch (e: Exception) {
+ *     // The lambda argument passed to this logger method has a LogBuilder as its receiver, which
+ *     // means that you can set `LogBuilder.cause` and call `LogBuilder.addField` in this scope.
+ *     log.error {
+ *       cause = e
+ *       addField("user", user)
+ *       "Failed to store user in database"
+ *     }
+ *   }
+ * }
+ * ```
+ */
 @JvmInline // Inline value class, since we just wrap a Logback logging event
 value class LogBuilder
 internal constructor(
@@ -20,14 +43,14 @@ internal constructor(
     get() = (logEvent.throwableProxy as? ThrowableProxy)?.throwable
 
   /**
-   * Adds a [log marker][LogMarker] with the given key and value to the log.
+   * Adds a [log field][LogField] (structured key-value data) to the log.
    *
    * The value is serialized using `kotlinx.serialization`, so if you pass an object here, you
    * should make sure it is annotated with [@Serializable][kotlinx.serialization.Serializable].
    * Alternatively, you can pass your own serializer for the value. If serialization fails, we fall
    * back to calling `toString()` on the value.
    *
-   * If you have a value that is already serialized, you should use [addRawJsonMarker] instead.
+   * If you have a value that is already serialized, you should use [addRawJsonField] instead.
    *
    * ### Example
    *
@@ -41,15 +64,15 @@ internal constructor(
    *   val user = User(id = 1, name = "John Doe")
    *
    *   log.info {
-   *     addMarker("user", user)
+   *     addField("user", user)
    *     "Registered new user"
    *   }
    * }
    *
    * @Serializable data class User(val id: Long, val name: String)
    * ```
    *
-   * This gives the following output using `logstash-logback-encoder`:
+   * This gives the following output (using `logstash-logback-encoder`):
    * ```json
    * {
    *   "message": "Registered new user",
@@ -61,18 +84,19 @@ internal constructor(
    * }
    * ```
    */
-  inline fun <reified ValueT> addMarker(
+  inline fun <reified ValueT> addField(
       key: String,
       value: ValueT,
       serializer: SerializationStrategy<ValueT>? = null,
   ) {
-    if (!markerKeyAdded(key)) {
-      logEvent.addMarker(createLogstashMarker(key, value, serializer))
+    if (!keyAdded(key)) {
+      logEvent.addMarker(createLogstashField(key, value, serializer))
     }
   }
 
   /**
-   * Adds a [log marker][LogMarker] with the given key and pre-serialized JSON value to the log.
+   * Adds a [log field][LogField] (structured key-value data) to the log, with the given
+   * pre-serialized JSON value.
    *
    * By default, this function checks that the given JSON string is actually valid JSON. The reason
    * for this is that giving raw JSON to our log encoder when it is not in fact valid JSON can break
@@ -90,59 +114,58 @@ internal constructor(
    *   val userJson = """{"id":1,"name":"John Doe"}"""
    *
    *   log.info {
-   *     addRawJsonMarker("user", userJson)
+   *     addRawJsonField("user", userJson)
    *     "Registered new user"
    *   }
    * }
    * ```
    *
-   * This gives the following output using `logstash-logback-encoder`:
+   * This gives the following output (using `logstash-logback-encoder`):
    * ```json
    * {"message":"Registered new user","user":{"id":1,"name":"John Doe"},/* ...timestamp etc. */}
    * ```
    */
-  fun addRawJsonMarker(key: String, json: String, validJson: Boolean = false) {
-    if (!markerKeyAdded(key)) {
-      logEvent.addMarker(createRawJsonLogstashMarker(key, json, validJson))
+  fun addRawJsonField(key: String, json: String, validJson: Boolean = false) {
+    if (!keyAdded(key)) {
+      logEvent.addMarker(createRawJsonLogstashField(key, json, validJson))
     }
   }
 
   /**
-   * Adds the given [log marker][LogMarker] to the log. This is useful when you have a previously
-   * constructed log marker, from the [marker]/[rawJsonMarker] functions.
-   * - If you want to create a new marker and add it to the log, you should instead call [addMarker]
-   * - If you want to add the marker to all logs within a scope, you should instead use
+   * Adds the given [log field][LogField] to the log. This is useful when you have a previously
+   * constructed field from the [field][dev.hermannm.devlog.field]/[rawJsonField] functions.
+   * - If you want to create a new field and add it to the log, you should instead call [addField]
+   * - If you want to add the field to all logs within a scope, you should instead use
    *   [withLoggingContext]
    */
-  fun addExistingMarker(marker: LogMarker) {
-    if (!markerKeyAdded(marker.logstashMarker.fieldName)) {
-      logEvent.addMarker(marker.logstashMarker)
+  fun addPreconstructedField(field: LogField) {
+    if (!keyAdded(field.key)) {
+      logEvent.addMarker(field.logstashField)
     }
   }
 
-  /** Adds log markers from [withLoggingContext]. */
-  internal fun addMarkersFromContext() {
+  /** Adds log fields from [withLoggingContext]. */
+  internal fun addFieldsFromContext() {
     // loggingContext will be null if withLoggingContext has not been called in this thread
-    val contextMarkers = loggingContext.get() ?: return
+    val contextFields = loggingContext.get() ?: return
 
-    // Add context markers in reverse, so newest marker shows first
-    contextMarkers.forEachReversed { logstashMarker ->
-      // Don't add marker keys that have already been added
-      if (!markerKeyAdded(logstashMarker.fieldName)) {
-        logEvent.addMarker(logstashMarker)
+    // Add context fields in reverse, so newest field shows first
+    contextFields.forEachReversed { logstashField ->
+      // Don't add fields with keys that have already been added
+      if (!keyAdded(logstashField.fieldName)) {
+        logEvent.addMarker(logstashField)
       }
     }
   }
 
   /**
    * Checks if the log [cause] exception (or any of its own cause exceptions) implements the
-   * [WithLogMarkers] interface, and if so, adds those markers.
+   * [WithLogFields] interface, and if so, adds those fields.
    */
-  internal fun addMarkersFromCauseException() {
+  internal fun addFieldsFromCauseException() {
     // The `cause` here is the log event cause exception. But this exception may itself have a
     // `cause` exception, and that may have another one, and so on. We want to go through all these
-    // exceptions to look for log markers, so we re-assign this local variable as we iterate
-    // through.
+    // exceptions to look for log field, so we re-assign this local variable as we iterate through.
     var exception = cause
     // Limit the depth of cause exceptions, so we don't expose ourselves to infinite loops.
     // This can happen if:
@@ -152,11 +175,11 @@ internal constructor(
     // We set max depth to 10, which should be high enough to not affect real users.
     var depth = 0
     while (exception != null && depth < 10) {
-      if (exception is WithLogMarkers) {
-        exception.logMarkers.forEach { marker ->
-          // Don't add marker keys that have already been added
-          if (!markerKeyAdded(marker.logstashMarker.fieldName)) {
-            logEvent.addMarker(marker.logstashMarker)
+      if (exception is WithLogFields) {
+        exception.logFields.forEach { field ->
+          // Don't add fields with keys that have already been added
+          if (!keyAdded(field.key)) {
+            logEvent.addMarker(field.logstashField)
           }
         }
       }
@@ -167,12 +190,12 @@ internal constructor(
   }
 
   @PublishedApi
-  internal fun markerKeyAdded(key: String): Boolean {
-    /** [LogbackEvent.markerList] can be null if no markers have been added yet. */
-    val markers = logEvent.markerList ?: return false
+  internal fun keyAdded(key: String): Boolean {
+    /** [LogbackEvent.markerList] can be null if no fields have been added yet. */
+    val addedFields = logEvent.markerList ?: return false
 
-    return markers.any { existingMarker ->
-      existingMarker is SingleFieldAppendingMarker && existingMarker.fieldName == key
+    return addedFields.any { logstashField ->
+      logstashField is SingleFieldAppendingMarker && logstashField.fieldName == key
     }
   }
 }