Signing and notarizing a Kotlin desktop app for macOS

In the stack post I said signing turned out to be a whole project, and that I'd write it up because I couldn't find a straight answer when I needed one. This is that writeup, for macOS. I'm starting from what code signing even is, because when I began I didn't really know, and every guide I found assumed I did. Leovidly is a Kotlin desktop app built with Compose Desktop, which adds a few wrinkles most guides skip, so I'll show the real config too. Windows gets its own post.

Why macOS makes you do any of this. When someone downloads a Mac app and opens it, macOS runs a check first. That check is Gatekeeper. Its job is to stop you running software from a developer macOS can't identify, since that's a common way malware gets in. If your app isn't signed and cleared by Apple, Gatekeeper shows a warning, or on recent macOS just refuses to open it. So the whole goal is to make Gatekeeper trust your app, so a normal person can double-click it and it just opens.

Signing and notarization are two different things. People say 'signing' for the whole process, but there are two separate steps, and it helps to keep them apart.

Code signing is a cryptographic stamp on your app. It proves two things: the app was built by a specific, identified developer, and it hasn't been changed since. You sign with a certificate, which is basically an identity Apple issued to you and vouches for.

Notarization is a second step. You upload your finished app to Apple, an automated system scans it for malware and checks it's signed correctly, and if it passes, Apple issues a small receipt called a notarization ticket. Gatekeeper wants both: a valid signature and that ticket.

What you need from Apple. All of it comes with the Apple Developer Program, the paid membership that's $99 a year:

• A Developer ID Application certificate. This is the identity you sign with. You generate it once from your developer account and it lives in your Mac's keychain.
• Your Team ID, a short code identifying your account.
• An app-specific password. Apple lets you create throwaway passwords tied to your account, so you never put your real Apple password in a script or a CI secret. You use one for notarization.

What 'the app' actually is. Worth knowing before we sign anything: on macOS an app isn't a single file. A .app is a folder with a set structure, and the executable, resources and libraries all live inside it. When you sign, you sign that whole bundle and everything in it. That matters in a minute.

Signing, and the entitlements the JVM forces on you. To sign for distribution, Apple requires the Hardened Runtime: a stricter mode that limits what your app can do while running, like loading unsigned code or writing to executable memory. Good for security, and a problem for anything on the JVM.

Here's why. The JVM compiles code while your app runs. That's the JIT (just-in-time compilation), and it means writing fresh machine code into memory and executing it, which is exactly what the hardened runtime blocks by default. So a freshly signed JVM app crashes the moment it starts the VM. The first time I hit it, it was an EXC_BREAKPOINT inside pthread_jit_write_protect_np, which tells you nothing.

The fix is an entitlements file. Entitlements are a list of specific permissions baked into the signature, each saying 'this app is allowed to do X.' A JVM app needs these:

<!-- runtime.entitlements -->
<plist version="1.0">
<dict>
    <key>com.apple.security.cs.allow-jit</key>
    <true/>
    <key>com.apple.security.cs.allow-unsigned-executable-memory</key>
    <true/>
    <key>com.apple.security.cs.disable-library-validation</key>
    <true/>
</dict>
</plist>

The first two let the JVM write and run code on the fly. disable-library-validation lets the app load native libraries someone other than you signed, which matters for the next section.

Leovidly is packaged with Compose Desktop, which turns the project into a real Mac .app and .dmg. The signing config lives in the Gradle build, in the nativeDistributions block, where you point at your entitlements and your certificate:

macOS {
    bundleID = "com.yourcompany.yourapp"
    entitlementsFile.set(file("src/macos/entitlements/app.entitlements"))
    runtimeEntitlementsFile.set(file("src/macos/entitlements/runtime.entitlements"))

    signing {
        sign.set(true)
        identity.set("Developer ID Application: Your Name (TEAMID)")
    }
    notarization {
        appleID.set(appleId)
        password.set(appSpecificPassword)
        teamID.set(teamId)
    }
}

The bundleID is your app's unique identifier in reverse-domain form. The identity string has to match your Developer ID certificate's name exactly.

The native libraries problem. The part that took longest. Leovidly bundles native libraries, like FFmpeg for video. These are chunks of compiled machine code for a specific platform (on macOS, .dylib files in Apple's Mach-O format), which is different from normal Java code, which is portable bytecode packed into JAR files. The problem is these native libraries ship inside the JARs, and notarization refuses to accept unsigned machine code hidden in there. Apple opens every JAR, finds the unsigned .dylib files, and rejects the whole app.

The fix is to pull those libraries out of the JARs before signing, move them where macOS expects signed code to live (the app's Contents/Frameworks), and sign each one. A couple resist even that. libjnijavacpp is the reliable troublemaker, so I wrap the codesign tool to ad-hoc sign the stubborn ones (ad-hoc means signed with no real identity, just enough to satisfy the loader):

#!/bin/bash
# Ad-hoc sign the native libs that break normal signing.
REAL_CODESIGN=/usr/bin/codesign.real
[[ -x "$REAL_CODESIGN" ]] || REAL_CODESIGN=/usr/bin/codesign
LAST_ARG="${@: -1}"
if [[ "$LAST_ARG" == *"libjnijavacpp"* ]]; then
  exec "$REAL_CODESIGN" -s - "$LAST_ARG"
fi
exec "$REAL_CODESIGN" "$@"

If your app is pure Kotlin or Java with no native pieces, ignore this whole section. If it isn't, this is where the time goes.

Sending it to Apple. With everything signed, you package the app into a DMG (the disk image people download) and send that to Apple. notarytool is Apple's command-line tool for it:

xcrun notarytool submit YourApp.dmg \
  --apple-id "$APPLE_ID" \
  --password "$APPLE_APP_SPECIFIC_PASSWORD" \
  --team-id "$APPLE_TEAM_ID" \
  --wait

--wait holds the terminal until Apple finishes, usually a couple of minutes. If it comes back Invalid, it won't tell you why inline. You ask for the log with the submission ID it gave you:

xcrun notarytool log <submission-id> \
  --apple-id "$APPLE_ID" \
  --password "$APPLE_APP_SPECIFIC_PASSWORD" \
  --team-id "$APPLE_TEAM_ID"

That log is where you find the one .dylib you forgot. When it passes, you staple the ticket onto the DMG. Stapling attaches the notarization receipt to the file itself, so the app is trusted even on a machine that's offline and can't check with Apple:

xcrun stapler staple YourApp.dmg

Doing it in CI. On your own Mac, all this uses the certificate already in your keychain (macOS's built-in secure store for passwords, keys and certificates). GitHub Actions gives you a clean machine with no keychain, so you build one and load your certificate into it. You keep the certificate as a base64 text secret and decode it back to a file at build time:

# Create and unlock a temporary keychain
security create-keychain -p "$KEYCHAIN_PASSWORD" signing.keychain-db
security set-keychain-settings -lut 21600 "$KEYCHAIN_PATH"   # do not auto-lock mid-build
security unlock-keychain -p "$KEYCHAIN_PASSWORD" "$KEYCHAIN_PATH"

# Import the certificate (kept as a base64 secret)
echo "$MACOS_CERTIFICATE_P12_BASE64" | base64 --decode -o cert.p12
security import cert.p12 -k "$KEYCHAIN_PATH" -P "$MACOS_CERTIFICATE_PASSWORD" -T /usr/bin/codesign

# The line everyone forgets:
security set-key-partition-list -S apple-tool:,apple:,codesign: -s -k "$KEYCHAIN_PASSWORD" "$KEYCHAIN_PATH"

security list-keychains -s "$KEYCHAIN_PATH" login.keychain-db

Two lines there earned their comments the hard way. set-key-partition-list is the one everyone forgets: without it, codesign can see your certificate but isn't allowed to use its private key without a GUI prompt, and CI has no GUI, so it fails with a useless permissions error. And the keychain auto-locks after a timeout. A long build runs past it, the keychain locks itself mid-way, and signing dies for no obvious reason, so you set a generous timeout.

Checking it actually worked. Before shipping, you can confirm the app will pass Gatekeeper:

spctl --assess -vv YourApp.app
# accepted
# source=Notarized Developer ID

If it says accepted and Notarized Developer ID, a user's Mac will open it without a fuss.

That's macOS, start to finish: signed, notarized, stapled and checked. It's an absurd amount of ceremony for something users are supposed to never notice, which is exactly why I wanted it in one place. Windows does all of it differently, with different tools and a different certificate story, and that's the next post.