diff --git a/README.md b/README.md index 8cb98f4..a1bed61 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,8 @@ > fixed and replied by human. A Fabric client mod that sends your Minecraft screenshots to [Immich](https://immich.app) -and lets you browse the album back inside the game. This mod also supports periodic screenshooting. +or [Google Photos](https://photos.google.com) and lets you browse the album back +inside the game. This mod also supports periodic screenshooting. Take a screenshot with F2 and it uploads in the background. Press F6 to see what is uploading, scroll through the album's timeline, or change how any of it @@ -73,6 +74,49 @@ clients or worlds stays tellable apart. Settings save when you close the screen. **Revert** undoes everything since you opened it. +## Connect it to Google Photos + +Press **F6** → **Settings** and switch **Provider** to Google Photos. The two +fields are an **OAuth client ID** and an **OAuth client secret**, and you have to +make them yourself — there is no key to paste from a Google Photos settings page, +and this mod cannot ship one for you. Google classes the Photos scopes as +restricted, so a shipped credential would need Google's verification, would name +this mod as the data controller for everyone's photos, and would break for +everybody at once the day it was revoked. + +Making one is a five-minute detour through the +[Google Cloud console](https://console.cloud.google.com), once: + +1. Create a project (any name). +2. **APIs & Services → Library**, find **Photos Library API**, press Enable. +3. **APIs & Services → OAuth consent screen**: pick External, fill in the three + required fields, and add your own Google address under **Test users**. The app + can stay unpublished — a test user is exactly what you are. +4. **APIs & Services → Credentials → Create credentials → OAuth client ID**, + application type **Desktop app**. Copy the client ID and client secret. + +Paste both into Settings and press **Test**. Your browser opens Google's consent +page, you allow it, and the tab tells you to go back to the game; the settings +screen then shows the account you signed in as. Nothing is typed into Minecraft +except the two client fields, and the sign-in that comes back is kept in +`google-photos.json` next to your settings, readable only by your user account. + +Three things are genuinely different from Immich, and all three are Google's +rules rather than choices made here: + +- **Browse only shows what PhotoSync uploaded.** Google withdrew the broad + library scopes in March 2025. No application can read your Google Photos + account any more, so the Browse tab is a view of this mod's own screenshots — + not of your holiday photos. +- **Albums have to be ones PhotoSync made.** The album picker lists those only, + because an upload into any other album is refused. Type a name and press Create + and it will be there. +- **Retries can leave a duplicate.** The Library API has no checksum, no + duplicate response, and no way to ask whether something is already stored. If + the game dies between an upload finishing and its confirmation arriving, the + retry stores a second copy. Immich does not have this problem; Google's API + gives nothing to solve it with. + ## The screens **Uploads** is the queue: everything still in flight plus the last hundred that @@ -148,7 +192,7 @@ Either way nothing is lost. The queue is on disk and is written through on every state change, so unfinished uploads resume the next time you play — the dialog exists to save you the wait, not to prevent a loss. An upload that was mid-flight when the game died is simply tried again, and Immich recognises the retry as a -duplicate rather than storing a second copy. +duplicate rather than storing a second copy. (Google Photos cannot; see above.) You can turn the dialog off under Settings → Uploads → *Ask before quitting mid-upload*, in which case the game gets a three-second grace period to finish @@ -232,10 +276,12 @@ the way it is, [`docs/PORTING.md`](docs/PORTING.md) is the document for that: it covers the bucket scheme, the measured API breakpoints between every supported version, and the procedure for a new release. -Adding a photo service other than Immich means implementing `PhotoProvider` and +Adding a third photo service means implementing `PhotoProvider` and `ProviderFactory` in `shared/core` and registering it with `ProviderCatalog`. No UI or platform code needs to change — the settings screen builds its connection -fields from what the provider declares. +fields from what the provider declares. `provider/immich` is the simpler of the +two worked examples; `provider/google` is the one to read if the service needs +OAuth rather than a key. ## Licence diff --git a/platform/1.20.1/src/main/java/dev/photosync/platform/impl/GameAdapter.java b/platform/1.20.1/src/main/java/dev/photosync/platform/impl/GameAdapter.java index 88fd6f8..d12a14b 100644 --- a/platform/1.20.1/src/main/java/dev/photosync/platform/impl/GameAdapter.java +++ b/platform/1.20.1/src/main/java/dev/photosync/platform/impl/GameAdapter.java @@ -91,6 +91,18 @@ public final class GameAdapter implements GameContext { Util.getPlatform().openFile(target.toFile()); } + /** + * Hands a URL to whatever the desktop opens links with. + * + *

Minecraft's own opener, the one behind every link in the game's menus, + * rather than {@code java.awt.Desktop}: on macOS the game holds the main + * thread for GLFW and an AWT call from here would never come back. + */ + @Override + public void openUrl(String url) { + Util.getPlatform().openUri(url); + } + @Override public String minecraftVersion() { return minecraftVersion; diff --git a/platform/1.20.4/src/main/java/dev/photosync/platform/impl/GameAdapter.java b/platform/1.20.4/src/main/java/dev/photosync/platform/impl/GameAdapter.java index 88fd6f8..d12a14b 100644 --- a/platform/1.20.4/src/main/java/dev/photosync/platform/impl/GameAdapter.java +++ b/platform/1.20.4/src/main/java/dev/photosync/platform/impl/GameAdapter.java @@ -91,6 +91,18 @@ public final class GameAdapter implements GameContext { Util.getPlatform().openFile(target.toFile()); } + /** + * Hands a URL to whatever the desktop opens links with. + * + *

Minecraft's own opener, the one behind every link in the game's menus, + * rather than {@code java.awt.Desktop}: on macOS the game holds the main + * thread for GLFW and an AWT call from here would never come back. + */ + @Override + public void openUrl(String url) { + Util.getPlatform().openUri(url); + } + @Override public String minecraftVersion() { return minecraftVersion; diff --git a/platform/1.20.6/src/main/java/dev/photosync/platform/impl/GameAdapter.java b/platform/1.20.6/src/main/java/dev/photosync/platform/impl/GameAdapter.java index 88fd6f8..d12a14b 100644 --- a/platform/1.20.6/src/main/java/dev/photosync/platform/impl/GameAdapter.java +++ b/platform/1.20.6/src/main/java/dev/photosync/platform/impl/GameAdapter.java @@ -91,6 +91,18 @@ public final class GameAdapter implements GameContext { Util.getPlatform().openFile(target.toFile()); } + /** + * Hands a URL to whatever the desktop opens links with. + * + *

Minecraft's own opener, the one behind every link in the game's menus, + * rather than {@code java.awt.Desktop}: on macOS the game holds the main + * thread for GLFW and an AWT call from here would never come back. + */ + @Override + public void openUrl(String url) { + Util.getPlatform().openUri(url); + } + @Override public String minecraftVersion() { return minecraftVersion; diff --git a/platform/1.21.1/src/main/java/dev/photosync/platform/impl/GameAdapter.java b/platform/1.21.1/src/main/java/dev/photosync/platform/impl/GameAdapter.java index 88fd6f8..d12a14b 100644 --- a/platform/1.21.1/src/main/java/dev/photosync/platform/impl/GameAdapter.java +++ b/platform/1.21.1/src/main/java/dev/photosync/platform/impl/GameAdapter.java @@ -91,6 +91,18 @@ public final class GameAdapter implements GameContext { Util.getPlatform().openFile(target.toFile()); } + /** + * Hands a URL to whatever the desktop opens links with. + * + *

Minecraft's own opener, the one behind every link in the game's menus, + * rather than {@code java.awt.Desktop}: on macOS the game holds the main + * thread for GLFW and an AWT call from here would never come back. + */ + @Override + public void openUrl(String url) { + Util.getPlatform().openUri(url); + } + @Override public String minecraftVersion() { return minecraftVersion; diff --git a/platform/1.21.11/src/main/java/dev/photosync/platform/impl/GameAdapter.java b/platform/1.21.11/src/main/java/dev/photosync/platform/impl/GameAdapter.java index d74564d..97d378c 100644 --- a/platform/1.21.11/src/main/java/dev/photosync/platform/impl/GameAdapter.java +++ b/platform/1.21.11/src/main/java/dev/photosync/platform/impl/GameAdapter.java @@ -91,6 +91,18 @@ public final class GameAdapter implements GameContext { Util.getPlatform().openFile(target.toFile()); } + /** + * Hands a URL to whatever the desktop opens links with. + * + *

Minecraft's own opener, the one behind every link in the game's menus, + * rather than {@code java.awt.Desktop}: on macOS the game holds the main + * thread for GLFW and an AWT call from here would never come back. + */ + @Override + public void openUrl(String url) { + Util.getPlatform().openUri(url); + } + @Override public String minecraftVersion() { return minecraftVersion; diff --git a/platform/1.21.4/src/main/java/dev/photosync/platform/impl/GameAdapter.java b/platform/1.21.4/src/main/java/dev/photosync/platform/impl/GameAdapter.java index 88fd6f8..d12a14b 100644 --- a/platform/1.21.4/src/main/java/dev/photosync/platform/impl/GameAdapter.java +++ b/platform/1.21.4/src/main/java/dev/photosync/platform/impl/GameAdapter.java @@ -91,6 +91,18 @@ public final class GameAdapter implements GameContext { Util.getPlatform().openFile(target.toFile()); } + /** + * Hands a URL to whatever the desktop opens links with. + * + *

Minecraft's own opener, the one behind every link in the game's menus, + * rather than {@code java.awt.Desktop}: on macOS the game holds the main + * thread for GLFW and an AWT call from here would never come back. + */ + @Override + public void openUrl(String url) { + Util.getPlatform().openUri(url); + } + @Override public String minecraftVersion() { return minecraftVersion; diff --git a/platform/1.21.5/src/main/java/dev/photosync/platform/impl/GameAdapter.java b/platform/1.21.5/src/main/java/dev/photosync/platform/impl/GameAdapter.java index 88fd6f8..d12a14b 100644 --- a/platform/1.21.5/src/main/java/dev/photosync/platform/impl/GameAdapter.java +++ b/platform/1.21.5/src/main/java/dev/photosync/platform/impl/GameAdapter.java @@ -91,6 +91,18 @@ public final class GameAdapter implements GameContext { Util.getPlatform().openFile(target.toFile()); } + /** + * Hands a URL to whatever the desktop opens links with. + * + *

Minecraft's own opener, the one behind every link in the game's menus, + * rather than {@code java.awt.Desktop}: on macOS the game holds the main + * thread for GLFW and an AWT call from here would never come back. + */ + @Override + public void openUrl(String url) { + Util.getPlatform().openUri(url); + } + @Override public String minecraftVersion() { return minecraftVersion; diff --git a/platform/1.21.8/src/main/java/dev/photosync/platform/impl/GameAdapter.java b/platform/1.21.8/src/main/java/dev/photosync/platform/impl/GameAdapter.java index 88fd6f8..d12a14b 100644 --- a/platform/1.21.8/src/main/java/dev/photosync/platform/impl/GameAdapter.java +++ b/platform/1.21.8/src/main/java/dev/photosync/platform/impl/GameAdapter.java @@ -91,6 +91,18 @@ public final class GameAdapter implements GameContext { Util.getPlatform().openFile(target.toFile()); } + /** + * Hands a URL to whatever the desktop opens links with. + * + *

Minecraft's own opener, the one behind every link in the game's menus, + * rather than {@code java.awt.Desktop}: on macOS the game holds the main + * thread for GLFW and an AWT call from here would never come back. + */ + @Override + public void openUrl(String url) { + Util.getPlatform().openUri(url); + } + @Override public String minecraftVersion() { return minecraftVersion; diff --git a/platform/26.2/src/main/java/dev/photosync/platform/impl/GameAdapter.java b/platform/26.2/src/main/java/dev/photosync/platform/impl/GameAdapter.java index fc274a0..3eabbf0 100644 --- a/platform/26.2/src/main/java/dev/photosync/platform/impl/GameAdapter.java +++ b/platform/26.2/src/main/java/dev/photosync/platform/impl/GameAdapter.java @@ -92,6 +92,18 @@ public final class GameAdapter implements GameContext { Util.getPlatform().openFile(target.toFile()); } + /** + * Hands a URL to whatever the desktop opens links with. + * + *

Minecraft's own opener, the one behind every link in the game's menus, + * rather than {@code java.awt.Desktop}: on macOS the game holds the main + * thread for GLFW and an AWT call from here would never come back. + */ + @Override + public void openUrl(String url) { + Util.getPlatform().openUri(url); + } + @Override public String minecraftVersion() { return minecraftVersion; diff --git a/shared/client/src/main/java/dev/photosync/client/PhotoSyncClient.java b/shared/client/src/main/java/dev/photosync/client/PhotoSyncClient.java index 554b373..08a4523 100644 --- a/shared/client/src/main/java/dev/photosync/client/PhotoSyncClient.java +++ b/shared/client/src/main/java/dev/photosync/client/PhotoSyncClient.java @@ -53,9 +53,15 @@ public final class PhotoSyncClient implements AutoCloseable { private final AutoCapture autoCapture; public PhotoSyncClient(ClientBridge bridge) { - this(bridge, new PhotoSync(new ConfigLocations( - bridge.game().configDirectory(), - bridge.game().userDataDirectory())), Theme.dark()); + this(bridge, new PhotoSync( + new ConfigLocations( + bridge.game().configDirectory(), + bridge.game().userDataDirectory()), + // How Google Photos gets the player in front of its consent + // page. Core cannot open a browser, and the sign-in is called + // from a worker, so this hops back to the render thread the way + // everything else that touches the game does. + url -> bridge.game().submit(() -> bridge.game().openUrl(url))), Theme.dark()); } /** Takes its collaborators explicitly so a test can drive it without a game. */ diff --git a/shared/client/src/main/resources/assets/photosync/lang/en_us.json b/shared/client/src/main/resources/assets/photosync/lang/en_us.json index 408632b..aaa80fb 100644 --- a/shared/client/src/main/resources/assets/photosync/lang/en_us.json +++ b/shared/client/src/main/resources/assets/photosync/lang/en_us.json @@ -126,5 +126,10 @@ "photosync.provider.immich.endpoint": "Server URL", "photosync.provider.immich.endpoint.hint": "https://photos.example.com", "photosync.provider.immich.secret": "API key", - "photosync.provider.immich.secret.hint": "Account settings -> API Keys" + "photosync.provider.immich.secret.hint": "Account settings -> API Keys", + "photosync.provider.googlephotos.name": "Google Photos", + "photosync.provider.googlephotos.endpoint": "OAuth client ID", + "photosync.provider.googlephotos.endpoint.hint": "Google Cloud -> Credentials -> Desktop app", + "photosync.provider.googlephotos.secret": "OAuth client secret", + "photosync.provider.googlephotos.secret.hint": "From the same client; Test connection signs you in" } diff --git a/shared/client/src/main/resources/assets/photosync/lang/zh_cn.json b/shared/client/src/main/resources/assets/photosync/lang/zh_cn.json index 4ea8a5b..b62b7eb 100644 --- a/shared/client/src/main/resources/assets/photosync/lang/zh_cn.json +++ b/shared/client/src/main/resources/assets/photosync/lang/zh_cn.json @@ -126,5 +126,10 @@ "photosync.provider.immich.endpoint": "服务器地址", "photosync.provider.immich.endpoint.hint": "https://photos.example.com", "photosync.provider.immich.secret": "API 密钥", - "photosync.provider.immich.secret.hint": "账户设置 -> API 密钥" + "photosync.provider.immich.secret.hint": "账户设置 -> API 密钥", + "photosync.provider.googlephotos.name": "Google 相册", + "photosync.provider.googlephotos.endpoint": "OAuth 客户端 ID", + "photosync.provider.googlephotos.endpoint.hint": "Google Cloud -> 凭据 -> 桌面应用", + "photosync.provider.googlephotos.secret": "OAuth 客户端密钥", + "photosync.provider.googlephotos.secret.hint": "同一客户端；点击“测试连接”完成登录" } diff --git a/shared/core/src/main/java/dev/photosync/core/PhotoSync.java b/shared/core/src/main/java/dev/photosync/core/PhotoSync.java index c840ef2..a201320 100644 --- a/shared/core/src/main/java/dev/photosync/core/PhotoSync.java +++ b/shared/core/src/main/java/dev/photosync/core/PhotoSync.java @@ -8,6 +8,8 @@ import dev.photosync.core.config.PhotoSyncConfig; import dev.photosync.core.provider.ProviderCatalog; import dev.photosync.core.provider.ProviderFactory; import dev.photosync.core.provider.ProviderSession; +import dev.photosync.core.provider.google.GooglePhotosProviderFactory; +import dev.photosync.core.provider.google.SignInPrompt; import dev.photosync.core.provider.immich.ImmichProviderFactory; import dev.photosync.core.thumbnail.ThumbnailLoader; import dev.photosync.core.timeline.TimelineBrowser; @@ -41,6 +43,7 @@ public final class PhotoSync implements AutoCloseable { private static final String CONFIG_FILE = "photosync.json"; private static final String QUEUE_FILE = "uploads.json"; + private static final String TOKEN_FILE = GooglePhotosProviderFactory.TOKEN_FILE; @Getter private final ConfigLocations locations; @@ -63,7 +66,7 @@ public final class PhotoSync implements AutoCloseable { /** The standard set of backends, stored in the given directory. */ public PhotoSync(Path directory) { - this(ConfigLocations.fixed(directory), List.of(new ImmichProviderFactory())); + this(ConfigLocations.fixed(directory)); } /** Takes the backend list explicitly so tests can run against a fake one. */ @@ -73,7 +76,35 @@ public final class PhotoSync implements AutoCloseable { /** The standard set of backends, with a switchable storage location. */ public PhotoSync(ConfigLocations locations) { - this(locations, List.of(new ImmichProviderFactory())); + this(locations, SignInPrompt.logging()); + } + + /** + * The standard set of backends, told how to put the player in front of a + * browser. + * + *

The prompt exists for Google Photos alone: it is the only backend whose + * credential cannot be typed into the settings screen. Without one the + * sign-in URL still appears in the log and still works, which is what the + * other constructors fall back to. + */ + public PhotoSync(ConfigLocations locations, SignInPrompt signIn) { + this(locations, standardProviders(locations, signIn)); + } + + /** + * Immich first, deliberately: {@code ProviderCatalog.preferred()} is the + * head of this list, and it is what a fresh install starts configured for. + */ + private static List standardProviders(ConfigLocations locations, SignInPrompt signIn) { + return List.of( + new ImmichProviderFactory(), + new GooglePhotosProviderFactory( + // Resolved on each use rather than captured: the settings + // directory can change while the game is running, and the + // token has to follow the settings it belongs to. + () -> locations.directory(locations.resolve()).resolve(TOKEN_FILE), + signIn)); } /** Takes the backend list explicitly so tests can run against a fake one. */ @@ -155,6 +186,10 @@ public final class PhotoSync implements AutoCloseable { } Files.createDirectories(to); moveFile(from.resolve(CONFIG_FILE), to.resolve(CONFIG_FILE)); + // The Google Photos sign-in travels with the settings it belongs to. + // Leaving it behind would silently sign the player out the next time + // they opened the settings screen. + moveFile(from.resolve(TOKEN_FILE), to.resolve(TOKEN_FILE)); config.relocate(to.resolve(CONFIG_FILE)); locations.writeMarker(next); activeLocation = next; diff --git a/shared/core/src/main/java/dev/photosync/core/provider/google/GoogleAuth.java b/shared/core/src/main/java/dev/photosync/core/provider/google/GoogleAuth.java new file mode 100644 index 0000000..0baebf2 --- /dev/null +++ b/shared/core/src/main/java/dev/photosync/core/provider/google/GoogleAuth.java @@ -0,0 +1,451 @@ +package dev.photosync.core.provider.google; + +import com.google.gson.JsonElement; +import com.google.gson.JsonObject; +import com.google.gson.JsonParser; +import dev.photosync.core.persistence.JsonFile; +import dev.photosync.core.provider.ProviderConnection; +import dev.photosync.core.provider.ProviderException; +import lombok.extern.slf4j.Slf4j; + +import java.io.IOException; +import java.net.SocketTimeoutException; +import java.net.URI; +import java.net.URLEncoder; +import java.net.http.HttpClient; +import java.net.http.HttpRequest; +import java.net.http.HttpResponse; +import java.nio.charset.StandardCharsets; +import java.nio.file.Path; +import java.security.MessageDigest; +import java.security.NoSuchAlgorithmException; +import java.security.SecureRandom; +import java.time.Duration; +import java.time.Instant; +import java.util.Base64; +import java.util.LinkedHashMap; +import java.util.Map; +import java.util.StringJoiner; +import java.util.function.Supplier; + +/** + * The OAuth half of talking to Google Photos: getting a token, keeping it, and + * getting a new one when it expires. + * + *

Google Photos has no API key. The only credential a player can hold is a + * refresh token, and the only way to mint one is to send them to Google in a + * browser and catch the redirect. So this class does three things: it runs that + * flow once ({@link #signIn()}), it stores what comes back next to the settings + * file, and from then on it quietly trades the refresh token for access tokens + * ({@link #bearer()}). + * + *

The split between those two entry points is the important design decision + * here. {@link #signIn()} can open a browser window; {@link #bearer()} never + * can. An upload retrying in the background must not throw a consent page over + * the game, so it fails with an authentication error instead and the player + * signs in when they choose to, from the settings screen. + * + *

PKCE is used even though this is a confidential client with a secret, + * because the secret is sitting in a config file on a player's machine and the + * loopback redirect is reachable by anything else running as that user. It + * costs two hashes. + */ +@Slf4j +final class GoogleAuth { + + private static final URI AUTHORIZATION_ENDPOINT = URI.create("https://accounts.google.com/o/oauth2/v2/auth"); + private static final URI TOKEN_ENDPOINT = URI.create("https://oauth2.googleapis.com/token"); + + private static final Duration CONNECT_TIMEOUT = Duration.ofSeconds(10); + private static final Duration REQUEST_TIMEOUT = Duration.ofSeconds(30); + /** How long the player gets to find the browser window and press Allow. */ + private static final Duration CONSENT_TIMEOUT = Duration.ofMinutes(5); + /** Renew early: an access token that expires halfway through an upload is a failed upload. */ + private static final Duration EXPIRY_MARGIN = Duration.ofMinutes(5); + /** What Google actually issues, used only when a response omits the field. */ + private static final long DEFAULT_LIFETIME_SECONDS = 3600; + + /** + * What PhotoSync asks Google for. + * + *

Google withdrew the broad library scopes in March 2025, so no app can + * read a user's whole Google Photos account any more. What is left happens + * to be exactly this mod's shape: permission to add media, and permission to + * read back the media it added. The consequence is worth being blunt about, + * because it surprises people -- the Browse tab shows the screenshots + * PhotoSync uploaded, not the player's holiday photos. + * + *

{@code openid} and the email scope are here only so that "Test + * connection" can name the account that was signed in. Without them the + * settings screen can say the token works but not whose it is, and a player + * with two Google accounts has no way to check they picked the right one. + */ + private static final String SCOPES = String.join(" ", + "openid", + "https://www.googleapis.com/auth/userinfo.email", + "https://www.googleapis.com/auth/photoslibrary.appendonly", + "https://www.googleapis.com/auth/photoslibrary.readonly.appcreateddata"); + + /** What is kept on disk between sessions. The client id is recorded so a changed one invalidates it. */ + private record StoredSignIn(String clientId, String refreshToken, String account) { + } + + private final HttpClient http; + private final SecureRandom random = new SecureRandom(); + private final Object lock = new Object(); + + private final String clientId; + private final String clientSecret; + private final Supplier tokenFile; + private final SignInPrompt prompt; + + private String accessToken; + private Instant expiresAt = Instant.EPOCH; + private String refreshToken; + private String account = ""; + private boolean loaded; + + GoogleAuth(ProviderConnection connection, Supplier tokenFile, SignInPrompt prompt) { + this.clientId = connection.endpoint(); + this.clientSecret = connection.secret(); + this.tokenFile = tokenFile; + this.prompt = prompt; + this.http = HttpClient.newBuilder() + .connectTimeout(CONNECT_TIMEOUT) + .followRedirects(HttpClient.Redirect.NORMAL) + .build(); + } + + /** + * The one connection pool for everything PhotoSync says to Google. + * + *

{@link GooglePhotosApi} borrows it rather than building a second: the + * token endpoint and the Library API are the same company's TLS termination, + * and a mod running inside a game should not hold two pools to reach it. + */ + HttpClient http() { + return http; + } + + /** Whoever was signed in, once anything has asked for a token. Empty before that. */ + String account() { + synchronized (lock) { + return account; + } + } + + /** + * An access token for an API call, without ever prompting. + * + *

Fails rather than signing in, because this is reached from uploads and + * thumbnail fetches -- work that happens while the player is doing something + * else, where a browser window opening unbidden would be alarming. + */ + String bearer() throws ProviderException { + synchronized (lock) { + String usable = unexpired(); + if (usable != null) { + return usable; + } + load(); + if (refreshToken == null) { + throw new ProviderException(ProviderException.Kind.AUTHENTICATION, + "PhotoSync is not signed in to Google Photos. " + + "Open the settings screen and use Test connection to sign in."); + } + return refresh(); + } + } + + /** + * Makes sure there is a working sign-in, running the browser flow if there + * is not, and answers which account it belongs to. + * + *

This is what "Test connection" calls. It is the only path that may + * block for minutes, and it is a direct answer to a button the player just + * pressed. + */ + String signIn() throws ProviderException { + synchronized (lock) { + load(); + if (refreshToken != null) { + try { + refresh(); + return account; + } catch (ProviderException e) { + if (e.kind() != ProviderException.Kind.AUTHENTICATION) { + throw e; + } + // Revoked from the Google account page, or the client id in + // the settings was changed. Either way, start over. + log.info("The stored Google Photos sign-in no longer works; asking for a new one"); + refreshToken = null; + } + } + authorize(); + return account; + } + } + + /** + * Throws away the current access token so the next {@link #bearer()} mints a + * fresh one. + * + *

For the case the expiry margin cannot cover: Google saying 401 to a + * token this class still believes in. A clock that is wrong, a laptop + * suspended mid-request, or a grant revoked and re-issued all look like + * that, and the answer to all three is to stop trusting what is cached. + */ + void forgetAccessToken() { + synchronized (lock) { + accessToken = null; + expiresAt = Instant.EPOCH; + } + } + + private String unexpired() { + boolean fresh = accessToken != null && Instant.now().isBefore(expiresAt.minus(EXPIRY_MARGIN)); + return fresh ? accessToken : null; + } + + // ----------------------------------------------------------------------- + // The browser flow + // ----------------------------------------------------------------------- + + private void authorize() throws ProviderException { + String verifier = randomToken(64); + String state = randomToken(24); + try (LoopbackReceiver receiver = new LoopbackReceiver()) { + String redirect = receiver.redirectUri(); + Map query = new LinkedHashMap<>(); + query.put("client_id", clientId); + query.put("redirect_uri", redirect); + query.put("response_type", "code"); + query.put("scope", SCOPES); + query.put("code_challenge", challengeFor(verifier)); + query.put("code_challenge_method", "S256"); + // Offline access is what makes Google issue a refresh token at all, + // and it only does so on a consent it has not already recorded -- + // hence the forced prompt, which is the difference between signing + // in once and signing in every hour. + query.put("access_type", "offline"); + query.put("prompt", "consent"); + query.put("state", state); + + String url = AUTHORIZATION_ENDPOINT + "?" + encode(query); + log.info("Waiting for a Google Photos sign-in from the browser on {}", redirect); + prompt.show(url); + + Map answer = receiver.awaitRedirect(CONSENT_TIMEOUT); + String refusal = answer.get("error"); + if (refusal != null) { + throw new ProviderException(ProviderException.Kind.AUTHENTICATION, + "Google would not sign PhotoSync in: " + refusal); + } + if (!state.equals(answer.get("state"))) { + // Somebody other than the browser we sent got to the port first. + throw new ProviderException(ProviderException.Kind.PROTOCOL, + "The sign-in that came back was not the one PhotoSync started"); + } + String code = answer.get("code"); + if (code == null || code.isBlank()) { + throw new ProviderException(ProviderException.Kind.PROTOCOL, + "Google's redirect carried no authorization code"); + } + + Map form = new LinkedHashMap<>(); + form.put("grant_type", "authorization_code"); + form.put("code", code); + form.put("client_id", clientId); + form.put("client_secret", clientSecret); + form.put("code_verifier", verifier); + form.put("redirect_uri", redirect); + adopt(exchange(form)); + log.info("Signed in to Google Photos as {}", account.isEmpty() ? "an unnamed account" : account); + } catch (SocketTimeoutException e) { + throw new ProviderException(ProviderException.Kind.AUTHENTICATION, + "The Google Photos sign-in was not finished in time. Press Test connection to try again.", e); + } catch (IOException e) { + throw new ProviderException(ProviderException.Kind.NETWORK, + "Could not listen for Google's answer: " + e.getMessage(), e); + } + } + + private String refresh() throws ProviderException { + Map form = new LinkedHashMap<>(); + form.put("grant_type", "refresh_token"); + form.put("refresh_token", refreshToken); + form.put("client_id", clientId); + form.put("client_secret", clientSecret); + adopt(exchange(form)); + return accessToken; + } + + /** Takes whatever a grant carried, keeping what it did not mention. */ + private void adopt(JsonObject grant) { + accessToken = text(grant, "access_token"); + long lifetime = grant.has("expires_in") + ? grant.get("expires_in").getAsLong() + : DEFAULT_LIFETIME_SECONDS; + expiresAt = Instant.now().plusSeconds(lifetime); + // A refresh response repeats neither the refresh token nor, always, the + // identity: absence means unchanged, not revoked. + String issued = text(grant, "refresh_token"); + if (issued != null) { + refreshToken = issued; + } + String email = emailIn(text(grant, "id_token")); + if (email != null) { + account = email; + } + store(); + } + + private JsonObject exchange(Map form) throws ProviderException { + HttpRequest request = HttpRequest.newBuilder(TOKEN_ENDPOINT) + .header("Content-Type", "application/x-www-form-urlencoded") + .header("Accept", "application/json") + .timeout(REQUEST_TIMEOUT) + .POST(HttpRequest.BodyPublishers.ofString(encode(form), StandardCharsets.UTF_8)) + .build(); + try { + HttpResponse response = http.send(request, HttpResponse.BodyHandlers.ofString()); + JsonObject body = objectIn(response.body()); + if (response.statusCode() / 100 == 2 && body.has("access_token")) { + return body; + } + throw failure(response.statusCode(), body); + } catch (IOException e) { + throw new ProviderException(ProviderException.Kind.NETWORK, + "Could not reach Google's token endpoint: " + e.getMessage(), e); + } catch (InterruptedException e) { + Thread.currentThread().interrupt(); + throw new ProviderException(ProviderException.Kind.NETWORK, "Interrupted while signing in to Google", e); + } + } + + private static ProviderException failure(int status, JsonObject body) { + String code = text(body, "error"); + String detail = text(body, "error_description"); + ProviderException.Kind kind; + if ("invalid_grant".equals(code) || "invalid_client".equals(code) || status == 401 || status == 403) { + // invalid_grant is what a revoked or expired refresh token looks + // like, and it is the one case where signing in again is the fix. + kind = ProviderException.Kind.AUTHENTICATION; + } else if (status >= 500) { + kind = ProviderException.Kind.SERVER; + } else if (status == 429) { + kind = ProviderException.Kind.RATE_LIMITED; + } else { + kind = ProviderException.Kind.PROTOCOL; + } + return new ProviderException(kind, "Google refused the sign-in (HTTP " + status + ", " + + (code == null ? "no details" : code) + (detail == null ? "" : ": " + detail) + ")"); + } + + // ----------------------------------------------------------------------- + // The token on disk + // ----------------------------------------------------------------------- + + private void load() { + if (loaded) { + return; + } + loaded = true; + JsonFile file = new JsonFile(tokenFile.get()); + JsonElement stored = file.readTree().orElse(null); + if (stored == null || !stored.isJsonObject()) { + return; + } + try { + StoredSignIn signIn = file.gson().fromJson(stored, StoredSignIn.class); + // A refresh token belongs to the OAuth client that minted it. If the + // player has pasted a different client id, this one is dead weight. + if (signIn == null || !clientId.equals(signIn.clientId())) { + return; + } + refreshToken = signIn.refreshToken() == null || signIn.refreshToken().isBlank() + ? null + : signIn.refreshToken(); + account = signIn.account() == null ? "" : signIn.account(); + } catch (RuntimeException e) { + log.warn("Could not read the Google Photos sign-in in {}; signing in again", file.path(), e); + } + } + + private void store() { + JsonFile file = new JsonFile(tokenFile.get()); + try { + file.write(file.gson().toJsonTree(new StoredSignIn(clientId, refreshToken, account))); + } catch (IOException e) { + // The sign-in still works for this session. It just will not survive + // a restart, which is worth a warning and not worth failing over. + log.warn("Could not save the Google Photos sign-in to {}", file.path(), e); + } + } + + // ----------------------------------------------------------------------- + + private String randomToken(int bytes) { + byte[] value = new byte[bytes]; + random.nextBytes(value); + return Base64.getUrlEncoder().withoutPadding().encodeToString(value); + } + + private static String challengeFor(String verifier) { + try { + MessageDigest sha256 = MessageDigest.getInstance("SHA-256"); + return Base64.getUrlEncoder().withoutPadding() + .encodeToString(sha256.digest(verifier.getBytes(StandardCharsets.US_ASCII))); + } catch (NoSuchAlgorithmException e) { + throw new IllegalStateException("Every JVM is required to provide SHA-256", e); + } + } + + /** + * The email claim out of an OpenID token, or null. + * + *

Read rather than verified: this is Google's own answer, arriving over + * TLS from Google's own token endpoint, and it is used for one thing -- + * putting a name next to "Test connection". Nothing is authorised by it. + */ + private static String emailIn(String idToken) { + if (idToken == null) { + return null; + } + String[] parts = idToken.split("\\."); + if (parts.length < 2) { + return null; + } + try { + JsonObject claims = objectIn(new String( + Base64.getUrlDecoder().decode(parts[1]), StandardCharsets.UTF_8)); + return text(claims, "email"); + } catch (RuntimeException e) { + log.debug("Could not read the account name out of Google's id token: {}", e.toString()); + return null; + } + } + + /** Whatever object is in {@code body}, or an empty one -- an unparseable answer is a missing answer. */ + private static JsonObject objectIn(String body) { + if (body == null || body.isBlank()) { + return new JsonObject(); + } + JsonElement parsed = JsonParser.parseString(body); + return parsed.isJsonObject() ? parsed.getAsJsonObject() : new JsonObject(); + } + + private static String text(JsonObject body, String member) { + JsonElement value = body.get(member); + return value == null || !value.isJsonPrimitive() ? null : value.getAsString(); + } + + private static String encode(Map values) { + StringJoiner joiner = new StringJoiner("&"); + values.forEach((name, value) -> joiner.add( + URLEncoder.encode(name, StandardCharsets.UTF_8) + "=" + + URLEncoder.encode(value, StandardCharsets.UTF_8))); + return joiner.toString(); + } +} diff --git a/shared/core/src/main/java/dev/photosync/core/provider/google/GoogleDtos.java b/shared/core/src/main/java/dev/photosync/core/provider/google/GoogleDtos.java new file mode 100644 index 0000000..f587e2d --- /dev/null +++ b/shared/core/src/main/java/dev/photosync/core/provider/google/GoogleDtos.java @@ -0,0 +1,93 @@ +package dev.photosync.core.provider.google; + +import java.util.List; + +/** + * The slices of the Google Photos Library API v1 that PhotoSync reads, mirrored + * as records for Gson. + * + *

The API is camelCase throughout, so these need no name mapping. Two of its + * conventions do need remembering, and both are why several fields below are + * {@code String} where a number would be expected: + * + *

+ * + *

Fields the mod does not use are simply absent; Gson ignores unknown JSON + * members, and the smaller this file is the fewer ways an API change breaks us. + */ +final class GoogleDtos { + + private GoogleDtos() { + } + + /** + * One photo or video. + * + *

{@code baseUrl} is the only way to get pixels, it is not a stable + * identifier, and it stops working roughly an hour after it was issued -- + * see {@code GooglePhotosProvider#thumbnail}. + */ + record MediaItem( + String id, + String description, + String baseUrl, + String mimeType, + String filename, + MediaMetadata mediaMetadata) { + } + + /** {@code creationTime} is RFC 3339 in UTC; the width and height are decimal strings. */ + record MediaMetadata(String creationTime, String width, String height) { + } + + record MediaItemsResponse(List mediaItems, String nextPageToken) { + } + + /** Body for {@code mediaItems:search}. Only set the fields being used; nulls are omitted by Gson. */ + record SearchRequest(String albumId, Integer pageSize, String pageToken) { + } + + record AlbumSummary(String id, String title, String mediaItemsCount, String coverPhotoMediaItemId) { + } + + record AlbumsResponse(List albums, String nextPageToken) { + } + + record CreateAlbumRequest(NewAlbum album) { + } + + record NewAlbum(String title) { + } + + record BatchCreateRequest(String albumId, List newMediaItems) { + } + + record NewMediaItem(String description, SimpleMediaItem simpleMediaItem) { + } + + /** {@code uploadToken} is what {@code /v1/uploads} returned, as plain text. */ + record SimpleMediaItem(String uploadToken, String fileName) { + } + + record BatchCreateResponse(List newMediaItemResults) { + } + + /** A per-item outcome: {@code status.code} 0 means this one was created. */ + record NewMediaItemResult(String uploadToken, Status status, MediaItem mediaItem) { + } + + record Status(Integer code, String message) { + } + + /** Google's standard error envelope, which every endpoint here shares. */ + record ApiError(ErrorBody error) { + } + + record ErrorBody(Integer code, String message, String status) { + } +} diff --git a/shared/core/src/main/java/dev/photosync/core/provider/google/GooglePhotosApi.java b/shared/core/src/main/java/dev/photosync/core/provider/google/GooglePhotosApi.java new file mode 100644 index 0000000..1346f7f --- /dev/null +++ b/shared/core/src/main/java/dev/photosync/core/provider/google/GooglePhotosApi.java @@ -0,0 +1,339 @@ +package dev.photosync.core.provider.google; + +import com.google.gson.Gson; +import com.google.gson.JsonSyntaxException; +import dev.photosync.core.provider.ProviderException; +import dev.photosync.core.provider.TransferCancelledException; +import dev.photosync.core.provider.TransferProgress; + +import java.io.FilterInputStream; +import java.io.IOException; +import java.io.InputStream; +import java.io.UncheckedIOException; +import java.lang.reflect.Type; +import java.net.URI; +import java.net.URLEncoder; +import java.net.http.HttpClient; +import java.net.http.HttpRequest; +import java.net.http.HttpResponse; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.time.Duration; +import java.util.Map; +import java.util.Optional; +import java.util.StringJoiner; + +/** + * Everything HTTP about talking to the Google Photos Library API: bearer + * tokens, timeouts, the upload protocol, and turning a status code into a + * {@link ProviderException} the upload queue can act on. + * + *

Kept apart from {@link GooglePhotosProvider} for the same reason Immich's + * is: the provider then reads as a list of API calls and their mapping onto + * PhotoSync's model, with no transport noise in between. + */ +final class GooglePhotosApi { + + private static final URI ROOT = URI.create("https://photoslibrary.googleapis.com"); + private static final Duration REQUEST_TIMEOUT = Duration.ofSeconds(30); + /** Uploads get their own budget: a screenshot over a slow uplink is not a hung server. */ + private static final Duration UPLOAD_TIMEOUT = Duration.ofMinutes(10); + /** Report progress at most this often, so a fast link is not drowned in callbacks. */ + private static final int PROGRESS_STRIDE_BYTES = 64 * 1024; + + private final HttpClient http; + private final GoogleAuth auth; + private final Gson gson = new Gson(); + + GooglePhotosApi(GoogleAuth auth) { + this.auth = auth; + this.http = auth.http(); + } + + T get(String path, Map query, Type type) throws ProviderException { + HttpRequest.Builder request = HttpRequest.newBuilder(uri(path, query)) + .header("Accept", "application/json") + .timeout(REQUEST_TIMEOUT) + .GET(); + return decode(send(request), type); + } + + T post(String path, Object body, Type type) throws ProviderException { + HttpRequest.Builder request = HttpRequest.newBuilder(uri(path, Map.of())) + .header("Accept", "application/json") + .header("Content-Type", "application/json") + .timeout(REQUEST_TIMEOUT) + .POST(HttpRequest.BodyPublishers.ofString(gson.toJson(body), StandardCharsets.UTF_8)); + return decode(send(request), type); + } + + /** + * Streams a file to {@code /v1/uploads} and returns the upload token that + * {@code mediaItems:batchCreate} turns into a real media item. + * + *

This endpoint is not JSON in either direction. The body is the raw file + * -- which is why it is streamed from disk rather than read into a byte + * array, on a heap the game is already competing for -- and the answer is + * the token as bare text with no envelope around it. + */ + String upload(Path file, String mimeType, TransferProgress progress) throws ProviderException { + long size; + try { + size = Files.size(file); + } catch (IOException e) { + throw new ProviderException(ProviderException.Kind.SOURCE_UNREADABLE, "Cannot read " + file, e); + } + HttpRequest.Builder request = HttpRequest.newBuilder(uri("/v1/uploads", Map.of())) + .header("Content-Type", "application/octet-stream") + .header("X-Goog-Upload-Content-Type", mimeType) + // "raw" is the whole file in one request. The alternative, + // "resumable", buys the ability to continue an interrupted + // upload -- worth having for a video, not for a screenshot, + // and it costs two extra round trips on every one of them. + .header("X-Goog-Upload-Protocol", "raw") + .timeout(UPLOAD_TIMEOUT) + .POST(streamOf(file, size, progress)); + + String token = new String(send(request), StandardCharsets.UTF_8).trim(); + if (token.isEmpty()) { + throw new ProviderException(ProviderException.Kind.PROTOCOL, + "Google accepted the upload but returned no upload token"); + } + return token; + } + + /** + * Fetches image bytes from a {@code baseUrl}. + * + *

Deliberately unauthenticated. These URLs carry their own signed + * credential -- Google's own samples put them straight into an + * {@code } -- and sending a bearer token to a host outside the API + * endpoint is a habit worth not having. + */ + byte[] fetch(String url) throws ProviderException { + HttpRequest request = HttpRequest.newBuilder(URI.create(url)) + .timeout(REQUEST_TIMEOUT) + .GET() + .build(); + try { + HttpResponse response = http.send(request, HttpResponse.BodyHandlers.ofByteArray()); + if (response.statusCode() / 100 == 2) { + return response.body(); + } + throw failure(response.statusCode(), response.body()); + } catch (IOException e) { + throw new ProviderException(ProviderException.Kind.NETWORK, + "Could not download the image from Google: " + e.getMessage(), e); + } catch (InterruptedException e) { + Thread.currentThread().interrupt(); + throw new ProviderException(ProviderException.Kind.NETWORK, "Interrupted while downloading from Google", e); + } + } + + // ----------------------------------------------------------------------- + + /** + * Sends with a bearer token, and once more with a fresh one if Google says + * the first was no good. + * + *

An access token lives an hour and {@link GoogleAuth} renews it five + * minutes early, so this should not happen -- but a clock that is wrong, a + * session suspended mid-request, or a token revoked and re-granted all land + * here, and re-asking once is cheaper than failing an upload the player + * would have to notice and retry. + */ + private byte[] send(HttpRequest.Builder builder) throws ProviderException { + try { + return send(builder.copy().header("Authorization", "Bearer " + auth.bearer()).build()); + } catch (ProviderException e) { + if (e.kind() != ProviderException.Kind.AUTHENTICATION) { + throw e; + } + auth.forgetAccessToken(); + return send(builder.copy().header("Authorization", "Bearer " + auth.bearer()).build()); + } + } + + private byte[] send(HttpRequest request) throws ProviderException { + try { + HttpResponse response = http.send(request, HttpResponse.BodyHandlers.ofByteArray()); + int status = response.statusCode(); + if (status >= 200 && status < 300) { + return response.body(); + } + throw failure(status, response.body()); + } catch (IOException e) { + // A cancelled transfer unwinds through the request body stream and + // the client hands it back wrapped. It is not a network failure and + // must not be retried, so dig it out before classifying anything. + TransferCancelledException cancelled = findCancellation(e); + if (cancelled != null) { + throw cancelled; + } + throw new ProviderException(ProviderException.Kind.NETWORK, + "Could not reach Google Photos: " + e.getMessage(), e); + } catch (UncheckedIOException e) { + throw new ProviderException(ProviderException.Kind.SOURCE_UNREADABLE, + "Could not read the file to upload: " + e.getMessage(), e); + } catch (InterruptedException e) { + Thread.currentThread().interrupt(); + throw new ProviderException(ProviderException.Kind.NETWORK, "Interrupted while waiting for Google", e); + } + } + + private ProviderException failure(int status, byte[] body) { + ProviderException.Kind kind = switch (status) { + case 401 -> ProviderException.Kind.AUTHENTICATION; + // 403 is overloaded here. Google returns it both for a token that + // may not do this and for a project that has run out of quota, and + // only the message tells them apart -- so the message decides + // whether the queue waits or gives up. + case 403 -> looksLikeQuota(body) + ? ProviderException.Kind.RATE_LIMITED + : ProviderException.Kind.AUTHENTICATION; + case 404 -> ProviderException.Kind.NOT_FOUND; + case 408, 429 -> ProviderException.Kind.RATE_LIMITED; + default -> status >= 500 ? ProviderException.Kind.SERVER : ProviderException.Kind.PROTOCOL; + }; + return new ProviderException(kind, "Google Photos returned HTTP " + status + " (" + describe(body) + ")"); + } + + private boolean looksLikeQuota(byte[] body) { + String status = messageOf(body).orElse(""); + return status.contains("quota") || status.contains("Quota") || status.contains("rate limit"); + } + + /** Google answers errors as JSON; fall back to the raw text when it does not. */ + private String describe(byte[] body) { + String text = new String(body, StandardCharsets.UTF_8).trim(); + return messageOf(body) + .filter(message -> !message.isBlank()) + .orElseGet(() -> text.isEmpty() ? "no details" : abbreviate(text)); + } + + private Optional messageOf(byte[] body) { + String text = new String(body, StandardCharsets.UTF_8).trim(); + if (!text.startsWith("{")) { + return Optional.empty(); + } + try { + GoogleDtos.ApiError error = gson.fromJson(text, GoogleDtos.ApiError.class); + return Optional.ofNullable(error) + .map(GoogleDtos.ApiError::error) + .map(GoogleDtos.ErrorBody::message); + } catch (JsonSyntaxException ignored) { + return Optional.empty(); + } + } + + private T decode(byte[] body, Type type) throws ProviderException { + try { + T value = gson.fromJson(new String(body, StandardCharsets.UTF_8), type); + if (value == null) { + throw new ProviderException(ProviderException.Kind.PROTOCOL, + "Google Photos returned an empty response"); + } + return value; + } catch (JsonSyntaxException e) { + throw new ProviderException(ProviderException.Kind.PROTOCOL, + "Google Photos returned something that is not the JSON we expected", e); + } + } + + private static URI uri(String path, Map query) { + StringBuilder url = new StringBuilder(ROOT.toString()).append(path); + if (!query.isEmpty()) { + StringJoiner joiner = new StringJoiner("&"); + query.forEach((key, value) -> joiner.add( + URLEncoder.encode(key, StandardCharsets.UTF_8) + "=" + + URLEncoder.encode(value, StandardCharsets.UTF_8))); + url.append('?').append(joiner); + } + return URI.create(url.toString()); + } + + /** + * The file as a request body with a real {@code Content-Length}, reporting + * bytes as the client pulls them. + * + *

The supplier may be called more than once -- a redirect replays the + * body -- so it opens a fresh stream each time rather than capturing one. + */ + private static HttpRequest.BodyPublisher streamOf(Path file, long size, TransferProgress progress) { + return HttpRequest.BodyPublishers.fromPublisher( + HttpRequest.BodyPublishers.ofInputStream(() -> { + try { + return new CountingStream(Files.newInputStream(file), progress, size); + } catch (IOException e) { + // The supplier cannot throw a checked exception; the + // HttpClient unwraps this back into one on send(). + throw new UncheckedIOException(e); + } + }), + size); + } + + /** + * Counts bytes as the HTTP client pulls them, and tells the queue. + * + *

The callback may throw -- that is how a cancelled upload unwinds -- so + * it is invoked where the exception can propagate out of {@code read}. + */ + private static final class CountingStream extends FilterInputStream { + + private final TransferProgress progress; + private final long total; + private long sent; + private long lastReported; + + CountingStream(InputStream delegate, TransferProgress progress, long total) { + super(delegate); + this.progress = progress; + this.total = total; + } + + @Override + public int read() throws IOException { + int value = super.read(); + if (value >= 0) { + advance(1); + } + return value; + } + + @Override + public int read(byte[] buffer, int offset, int length) throws IOException { + int count = super.read(buffer, offset, length); + if (count > 0) { + advance(count); + } + return count; + } + + private void advance(int count) { + sent += count; + if (sent - lastReported >= PROGRESS_STRIDE_BYTES || sent == total) { + lastReported = sent; + progress.onProgress(sent, total); + } + } + } + + private static TransferCancelledException findCancellation(Throwable thrown) { + for (Throwable cause = thrown; cause != null; cause = cause.getCause()) { + if (cause instanceof TransferCancelledException cancelled) { + return cancelled; + } + if (cause.getCause() == cause) { + break; + } + } + return null; + } + + private static String abbreviate(String text) { + String flat = text.replaceAll("\\s+", " "); + return flat.length() <= 160 ? flat : flat.substring(0, 157) + "..."; + } +} diff --git a/shared/core/src/main/java/dev/photosync/core/provider/google/GooglePhotosProvider.java b/shared/core/src/main/java/dev/photosync/core/provider/google/GooglePhotosProvider.java new file mode 100644 index 0000000..d14fd2e --- /dev/null +++ b/shared/core/src/main/java/dev/photosync/core/provider/google/GooglePhotosProvider.java @@ -0,0 +1,505 @@ +package dev.photosync.core.provider.google; + +import dev.photosync.core.provider.Album; +import dev.photosync.core.provider.AlbumRef; +import dev.photosync.core.provider.AssetKind; +import dev.photosync.core.provider.BucketPage; +import dev.photosync.core.provider.PhotoProvider; +import dev.photosync.core.provider.ProviderConnection; +import dev.photosync.core.provider.ProviderDescriptor; +import dev.photosync.core.provider.ProviderException; +import dev.photosync.core.provider.ProviderIdentity; +import dev.photosync.core.provider.RemoteAsset; +import dev.photosync.core.provider.ThumbnailSize; +import dev.photosync.core.provider.TimelineBucket; +import dev.photosync.core.provider.TransferProgress; +import dev.photosync.core.provider.UploadReceipt; +import dev.photosync.core.provider.UploadRequest; +import lombok.extern.slf4j.Slf4j; + +import java.nio.file.Files; +import java.nio.file.Path; +import java.time.Duration; +import java.time.Instant; +import java.time.LocalDate; +import java.time.LocalDateTime; +import java.time.ZoneId; +import java.time.format.DateTimeParseException; +import java.util.ArrayList; +import java.util.Comparator; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Locale; +import java.util.Map; +import java.util.Optional; +import java.util.concurrent.ConcurrentHashMap; +import java.util.function.Supplier; + +/** + * Google Photos, expressed in PhotoSync's terms. + * + *

Three things about this API shape everything below, and all three are + * worth knowing before reading further. + * + *

It only ever shows you what this mod uploaded. Google withdrew the + * broad library scopes in March 2025. What remains is "add media" and "read + * back the media you added", so the Browse tab is a view of PhotoSync's own + * uploads -- not of the player's Google Photos account. That is a permission + * boundary, not a setting. + * + *

There is no bucket endpoint. Immich hands over every month and its + * count in one cheap call, which is what makes its timeline a true virtual + * list. Google has nothing of the kind, so {@link #timeline} pages through the + * media items itself and groups them into months, then keeps the result so + * {@link #page} is free. The cost is bounded by {@link #MAX_SCAN_PAGES}; past + * that the timeline is truncated and says so in the log. + * + *

Image URLs expire. Nothing here is a stable image address. Each + * media item carries a {@code baseUrl} that is good for about an hour and has + * the wanted dimensions appended to it, so this class caches those URLs and + * refetches the item when one stops working. + */ +@Slf4j +public final class GooglePhotosProvider implements PhotoProvider { + + /** Google's maximum for both listing endpoints. Fewer pages is fewer round trips. */ + private static final int SCAN_PAGE_SIZE = 100; + /** A ceiling on what one Browse tab costs: 50 pages of 100 is 5000 items. */ + private static final int MAX_SCAN_PAGES = 50; + /** How long a scan is reused before the timeline is walked again. */ + private static final Duration SCAN_LIFETIME = Duration.ofMinutes(2); + /** + * How long a {@code baseUrl} is trusted. Google documents roughly 60 minutes; + * this leaves a margin so a URL does not expire between being handed out and + * being fetched. + */ + private static final Duration BASE_URL_LIFETIME = Duration.ofMinutes(45); + + /** The longest side to ask for, per {@link ThumbnailSize}. */ + private static final int GRID_PIXELS = 512; + private static final int DETAIL_PIXELS = 2048; + + /** Google rejects a longer one, and the queue would rather truncate than fail. */ + private static final int MAX_DESCRIPTION = 1000; + + private final ProviderDescriptor descriptor; + private final GoogleAuth auth; + private final GooglePhotosApi api; + + /** {@code baseUrl}s learned from scans and item fetches, by media item id. */ + private final Map baseUrls = new ConcurrentHashMap<>(); + + private final Object scanLock = new Object(); + private Scan scan; + + GooglePhotosProvider(ProviderDescriptor descriptor, + ProviderConnection connection, + Supplier tokenFile, + SignInPrompt prompt) { + this.descriptor = descriptor; + this.auth = new GoogleAuth(connection, tokenFile, prompt); + this.api = new GooglePhotosApi(auth); + } + + @Override + public ProviderDescriptor descriptor() { + return descriptor; + } + + /** + * Signs in if that has not happened yet, then names the account. + * + *

This is the one method allowed to open a browser, and it is reached + * only from the "Test connection" button -- so consent appears because the + * player just asked for it, never in the middle of a game. Everything else + * goes through {@link GoogleAuth#bearer()}, which fails rather than prompts. + */ + @Override + public ProviderIdentity identify() throws ProviderException { + String account = auth.signIn(); + return new ProviderIdentity("Google Photos Library API v1", + account.isEmpty() ? "signed in" : account); + } + + /** + * Albums this mod created. + * + *

{@code excludeNonAppCreatedData} is not a filter for tidiness: the + * appendonly scope cannot add media to an album it did not create, so + * offering the player's other albums would be offering uploads that are + * going to be refused. + */ + @Override + public List albums() throws ProviderException { + List albums = new ArrayList<>(); + String pageToken = null; + int pages = 0; + do { + Map query = new LinkedHashMap<>(); + query.put("pageSize", "50"); + query.put("excludeNonAppCreatedData", "true"); + if (pageToken != null) { + query.put("pageToken", pageToken); + } + GoogleDtos.AlbumsResponse response = + api.get("/v1/albums", query, GoogleDtos.AlbumsResponse.class); + if (response.albums() != null) { + for (GoogleDtos.AlbumSummary summary : response.albums()) { + if (summary.id() == null) { + continue; + } + albums.add(new Album( + summary.id(), + Optional.ofNullable(summary.title()).orElse(summary.id()), + (int) Math.min(Integer.MAX_VALUE, number(summary.mediaItemsCount(), 0)), + Optional.ofNullable(summary.coverPhotoMediaItemId()))); + } + } + pageToken = response.nextPageToken(); + pages++; + } while (pageToken != null && !pageToken.isBlank() && pages < MAX_SCAN_PAGES); + + return albums.stream() + .sorted(Comparator.comparing(Album::name, String.CASE_INSENSITIVE_ORDER)) + .toList(); + } + + @Override + public Album createAlbum(String name) throws ProviderException { + GoogleDtos.AlbumSummary created = api.post("/v1/albums", + new GoogleDtos.CreateAlbumRequest(new GoogleDtos.NewAlbum(name)), + GoogleDtos.AlbumSummary.class); + if (created.id() == null) { + throw new ProviderException(ProviderException.Kind.PROTOCOL, + "Google Photos created an album without an id"); + } + return new Album(created.id(), name, 0, Optional.empty()); + } + + /** + * Uploads in Google's two steps: the bytes go to {@code /v1/uploads}, which + * answers with a token, and the token becomes a real media item through + * {@code mediaItems:batchCreate}. + * + *

The idempotency the interface asks for cannot be delivered here. + * {@link PhotoProvider#upload} requires that re-sending bytes the backend + * already holds come back as {@link UploadReceipt.Outcome#DUPLICATE}, and + * Google's API has no such concept: there is no checksum header, no + * duplicate status, and no way to ask whether an item exists. (The Google + * Photos app deduplicates its own backups; the API does not expose it.) So + * every upload that succeeds is reported as {@code CREATED}, and a job the + * queue retries because the first response was lost will leave two copies in + * the account. Nothing in this class can prevent that -- it is worth knowing + * rather than working around. + */ + @Override + public UploadReceipt upload(UploadRequest request, TransferProgress progress) throws ProviderException { + Path file = request.file(); + if (!Files.isReadable(file)) { + throw new ProviderException(ProviderException.Kind.SOURCE_UNREADABLE, "Cannot read " + file); + } + + String uploadToken = api.upload(file, mediaTypeOf(request.fileName()), progress); + + String description = request.description().length() > MAX_DESCRIPTION + ? request.description().substring(0, MAX_DESCRIPTION) + : request.description(); + GoogleDtos.BatchCreateRequest body = new GoogleDtos.BatchCreateRequest( + request.album().id().orElse(null), + List.of(new GoogleDtos.NewMediaItem( + description.isBlank() ? null : description, + new GoogleDtos.SimpleMediaItem(uploadToken, request.fileName())))); + + GoogleDtos.BatchCreateResponse response = + api.post("/v1/mediaItems:batchCreate", body, GoogleDtos.BatchCreateResponse.class); + + List results = response.newMediaItemResults(); + if (results == null || results.isEmpty()) { + throw new ProviderException(ProviderException.Kind.PROTOCOL, + "Google Photos accepted the upload but said nothing about what it did with it"); + } + GoogleDtos.NewMediaItemResult result = results.get(0); + + // batchCreate answers 200 even when the item inside it failed; the + // per-item status is the real outcome. Code 0 is OK in Google's status + // model, and an absent status means the same. + int code = result.status() == null || result.status().code() == null ? 0 : result.status().code(); + if (code != 0 || result.mediaItem() == null || result.mediaItem().id() == null) { + String detail = result.status() == null || result.status().message() == null + ? "no reason given" + : result.status().message(); + throw new ProviderException(ProviderException.Kind.PROTOCOL, + "Google Photos would not store the upload: " + detail); + } + + remember(result.mediaItem()); + // Always CREATED -- see the note above. Reporting DUPLICATE would be a lie. + return new UploadReceipt(result.mediaItem().id(), UploadReceipt.Outcome.CREATED); + } + + @Override + public List timeline(AlbumRef album) throws ProviderException { + return scanFor(album).buckets(); + } + + @Override + public BucketPage page(AlbumRef album, TimelineBucket bucket) throws ProviderException { + List assets = scanFor(album).pages().get(bucket.key()); + return new BucketPage(bucket, assets == null ? List.of() : assets); + } + + /** + * Image bytes at roughly the requested size. + * + *

Google serves these from a {@code baseUrl} with the wanted box appended + * -- {@code =w512-h512} means "fit inside 512 by 512" -- and always as JPEG, + * which is what the game's decoder wants. Videos have no separate thumbnail + * endpoint; the same URL without a {@code =dv} suffix returns a still frame, + * so a video tile needs no special case. + * + *

A {@code baseUrl} stops working roughly an hour after it was issued, + * and a player can leave the Browse tab open longer than that. So an expiry + * is not an error: the URL is dropped, the media item is fetched again for a + * fresh one, and the request is made once more. + */ + @Override + public byte[] thumbnail(String assetId, ThumbnailSize size) throws ProviderException { + int pixels = size == ThumbnailSize.DETAIL ? DETAIL_PIXELS : GRID_PIXELS; + String url = baseUrlOf(assetId, false); + try { + return api.fetch(sized(url, pixels)); + } catch (ProviderException e) { + boolean stale = e.kind() == ProviderException.Kind.NOT_FOUND + || e.kind() == ProviderException.Kind.AUTHENTICATION; + if (!stale) { + throw e; + } + log.debug("The image URL for {} has expired; asking Google for a new one", assetId); + return api.fetch(sized(baseUrlOf(assetId, true), pixels)); + } + } + + @Override + public void close() { + // Nothing to release: java.net.http.HttpClient has no close() before + // Java 21, and this module compiles to 17. It uses daemon threads and is + // collected with the provider. + } + + // ----------------------------------------------------------------------- + // The synthesised timeline + // ----------------------------------------------------------------------- + + /** + * One walk of the media items, grouped into months. + * + *

Held whole rather than per month because that is how it was fetched: + * Google's paging has no way to ask for "June", so once the pages have been + * read there is no reason to make {@link #page} pay for them again. + */ + private record Scan(AlbumRef album, + Instant takenAt, + List buckets, + Map> pages) { + } + + private Scan scanFor(AlbumRef album) throws ProviderException { + synchronized (scanLock) { + Scan current = scan; + boolean usable = current != null + && current.album().equals(album) + && Instant.now().isBefore(current.takenAt().plus(SCAN_LIFETIME)); + if (usable) { + return current; + } + Scan fresh = rescan(album); + scan = fresh; + return fresh; + } + } + + private Scan rescan(AlbumRef album) throws ProviderException { + List items = allItems(album); + + Map> pages = new LinkedHashMap<>(); + for (GoogleDtos.MediaItem item : items) { + RemoteAsset asset = assetOf(item); + if (asset == null) { + continue; + } + remember(item); + pages.computeIfAbsent(monthKey(asset.localDay()), key -> new ArrayList<>()).add(asset); + } + + List buckets = new ArrayList<>(pages.size()); + for (Map.Entry> entry : pages.entrySet()) { + entry.getValue().sort(Comparator.comparing(RemoteAsset::localCapturedAt).reversed()); + buckets.add(new TimelineBucket(entry.getKey(), LocalDate.parse(entry.getKey() + "-01"), + entry.getValue().size())); + } + buckets.sort(Comparator.naturalOrder()); + + log.debug("Google Photos timeline: {} items in {} months", items.size(), buckets.size()); + return new Scan(album, Instant.now(), List.copyOf(buckets), pages); + } + + /** + * Every media item in scope, up to the scan ceiling. + * + *

Note the asymmetry in what truncation costs. The library listing comes + * back newest first, so a cut-off drops the oldest items -- the ones furthest + * down the timeline. An album search comes back in album order, so a cut-off + * there drops the newest. Both are logged; only the second is likely to be + * noticed, and only by someone with more than {@link #MAX_SCAN_PAGES} pages + * of screenshots in one album. + */ + private List allItems(AlbumRef album) throws ProviderException { + List items = new ArrayList<>(); + String pageToken = null; + int pages = 0; + do { + GoogleDtos.MediaItemsResponse response; + if (album.isLibrary()) { + Map query = new LinkedHashMap<>(); + query.put("pageSize", Integer.toString(SCAN_PAGE_SIZE)); + if (pageToken != null) { + query.put("pageToken", pageToken); + } + response = api.get("/v1/mediaItems", query, GoogleDtos.MediaItemsResponse.class); + } else { + response = api.post("/v1/mediaItems:search", + new GoogleDtos.SearchRequest(album.id().orElseThrow(), SCAN_PAGE_SIZE, pageToken), + GoogleDtos.MediaItemsResponse.class); + } + if (response.mediaItems() != null) { + items.addAll(response.mediaItems()); + } + pageToken = response.nextPageToken(); + pages++; + } while (pageToken != null && !pageToken.isBlank() && pages < MAX_SCAN_PAGES); + + if (pageToken != null && !pageToken.isBlank()) { + log.info("Showing the first {} Google Photos items; there are more than PhotoSync browses in one go", + items.size()); + } + return items; + } + + private static RemoteAsset assetOf(GoogleDtos.MediaItem item) { + if (item.id() == null || item.mediaMetadata() == null) { + return null; + } + Instant capturedAt = instantOf(item.mediaMetadata().creationTime()); + if (capturedAt == null) { + // Without a timestamp the asset has no day to live under. + return null; + } + long width = number(item.mediaMetadata().width(), 0); + long height = number(item.mediaMetadata().height(), 0); + boolean video = item.mimeType() != null && item.mimeType().startsWith("video/"); + return new RemoteAsset( + item.id(), + video ? AssetKind.VIDEO : AssetKind.IMAGE, + capturedAt, + localTimeOf(capturedAt), + height > 0 ? (double) width / height : 1.0, + // Google has no ThumbHash equivalent, and computing one would + // mean downloading every image first -- exactly what the blurred + // placeholder exists to avoid. Tiles fill in as they load. + Optional.empty(), + Duration.ZERO); + } + + /** + * The capture instant read in the viewer's own time zone. + * + *

{@link RemoteAsset} wants the wall clock where the photo was taken, and + * Google -- unlike Immich -- returns no UTC offset to reconstruct it with. + * The player's zone is the closest available answer and is the right one in + * the ordinary case, where the screenshots were taken on this machine. + * Someone browsing from another continent will see days grouped by the + * calendar they are reading them on. + */ + private static LocalDateTime localTimeOf(Instant instant) { + return LocalDateTime.ofInstant(instant, ZoneId.systemDefault()); + } + + private static String monthKey(LocalDate day) { + return String.format(Locale.ROOT, "%04d-%02d", day.getYear(), day.getMonthValue()); + } + + // ----------------------------------------------------------------------- + // Image URLs + // ----------------------------------------------------------------------- + + private record CachedUrl(String url, Instant fetchedAt) { + } + + private void remember(GoogleDtos.MediaItem item) { + if (item.id() != null && item.baseUrl() != null && !item.baseUrl().isBlank()) { + baseUrls.put(item.id(), new CachedUrl(item.baseUrl(), Instant.now())); + } + } + + private String baseUrlOf(String assetId, boolean force) throws ProviderException { + if (!force) { + CachedUrl cached = baseUrls.get(assetId); + if (cached != null && Instant.now().isBefore(cached.fetchedAt().plus(BASE_URL_LIFETIME))) { + return cached.url(); + } + } + GoogleDtos.MediaItem item = + api.get("/v1/mediaItems/" + assetId, Map.of(), GoogleDtos.MediaItem.class); + if (item.baseUrl() == null || item.baseUrl().isBlank()) { + throw new ProviderException(ProviderException.Kind.NOT_FOUND, + "Google Photos has no image for " + assetId); + } + remember(item); + return item.baseUrl(); + } + + /** Google's sizing suffix: fit the image inside a box this many pixels on a side. */ + private static String sized(String baseUrl, int pixels) { + return baseUrl + "=w" + pixels + "-h" + pixels; + } + + // ----------------------------------------------------------------------- + + private static Instant instantOf(String rfc3339) { + if (rfc3339 == null) { + return null; + } + try { + return Instant.parse(rfc3339); + } catch (DateTimeParseException e) { + log.debug("Ignoring a Google Photos item with an unreadable creation time: {}", rfc3339); + return null; + } + } + + /** Google JSON-encodes its 64-bit integers as strings; this is the way back. */ + private static long number(String value, long fallback) { + if (value == null || value.isBlank()) { + return fallback; + } + try { + return Long.parseLong(value.trim()); + } catch (NumberFormatException e) { + return fallback; + } + } + + private static String mediaTypeOf(String fileName) { + String lower = fileName.toLowerCase(Locale.ROOT); + if (lower.endsWith(".png")) { + return "image/png"; + } + if (lower.endsWith(".jpg") || lower.endsWith(".jpeg")) { + return "image/jpeg"; + } + if (lower.endsWith(".webp")) { + return "image/webp"; + } + return "application/octet-stream"; + } +} diff --git a/shared/core/src/main/java/dev/photosync/core/provider/google/GooglePhotosProviderFactory.java b/shared/core/src/main/java/dev/photosync/core/provider/google/GooglePhotosProviderFactory.java new file mode 100644 index 0000000..fcad3da --- /dev/null +++ b/shared/core/src/main/java/dev/photosync/core/provider/google/GooglePhotosProviderFactory.java @@ -0,0 +1,54 @@ +package dev.photosync.core.provider.google; + +import dev.photosync.core.provider.PhotoProvider; +import dev.photosync.core.provider.ProviderConnection; +import dev.photosync.core.provider.ProviderDescriptor; +import dev.photosync.core.provider.ProviderFactory; +import dev.photosync.core.provider.ProviderId; + +import java.nio.file.Path; +import java.util.function.Supplier; + +/** + * Registers Google Photos with the {@code ProviderCatalog}. + * + *

Two things have to be handed in that Immich's factory does not need, and + * both are consequences of OAuth. The sign-in produces a refresh token that has + * to outlive the session, so this needs to know where to keep it; and getting + * that token means opening a browser, which core cannot do, so the caller + * supplies the way to do it. Both arrive as functions rather than values + * because the settings directory can move while the game is running. + */ +public final class GooglePhotosProviderFactory implements ProviderFactory { + + /** + * Where the refresh token lives, next to the settings file. + * + *

Kept out of {@code photosync.json} deliberately. That file is the one + * players paste into issue reports, and a refresh token in it is a + * standing invitation to somebody else's photo library. This one is written + * through {@code JsonFile}, which restricts it to the owning user. + */ + public static final String TOKEN_FILE = "google-photos.json"; + + private final ProviderDescriptor descriptor = + new ProviderDescriptor(new ProviderId("googlephotos"), "photosync.provider.googlephotos", true); + + private final Supplier tokenFile; + private final SignInPrompt prompt; + + public GooglePhotosProviderFactory(Supplier tokenFile, SignInPrompt prompt) { + this.tokenFile = tokenFile; + this.prompt = prompt; + } + + @Override + public ProviderDescriptor descriptor() { + return descriptor; + } + + @Override + public PhotoProvider connect(ProviderConnection connection) { + return new GooglePhotosProvider(descriptor, connection, tokenFile, prompt); + } +} diff --git a/shared/core/src/main/java/dev/photosync/core/provider/google/LoopbackReceiver.java b/shared/core/src/main/java/dev/photosync/core/provider/google/LoopbackReceiver.java new file mode 100644 index 0000000..b0cbbc3 --- /dev/null +++ b/shared/core/src/main/java/dev/photosync/core/provider/google/LoopbackReceiver.java @@ -0,0 +1,149 @@ +package dev.photosync.core.provider.google; + +import lombok.extern.slf4j.Slf4j; + +import java.io.BufferedReader; +import java.io.IOException; +import java.io.InputStreamReader; +import java.io.OutputStream; +import java.net.InetAddress; +import java.net.ServerSocket; +import java.net.Socket; +import java.net.SocketTimeoutException; +import java.net.URLDecoder; +import java.nio.charset.StandardCharsets; +import java.time.Duration; +import java.util.LinkedHashMap; +import java.util.Map; + +/** + * The other half of Google's redirect: a socket on the loopback interface that + * waits for the browser to come back with an authorization code. + * + *

A raw {@link ServerSocket} rather than {@code com.sun.net.httpserver}. + * Minecraft ships its own trimmed Java runtime, {@code jdk.httpserver} is not a + * module we can prove is in it on every launcher, and a sign-in that throws + * {@code NoClassDefFoundError} on somebody else's machine is not worth the + * convenience. One request line and one fixed response is all this needs. + * + *

Google allows a desktop client to redirect to {@code 127.0.0.1} on any + * port without registering it, which is why the port can be whatever the OS + * hands out. + */ +@Slf4j +final class LoopbackReceiver implements AutoCloseable { + + /** Long enough for a browser that is still starting up, short enough not to hang a worker. */ + private static final int READ_TIMEOUT_MILLIS = 10_000; + + private final ServerSocket socket; + + LoopbackReceiver() throws IOException { + // Loopback only, and a backlog of one: this listens for a single + // redirect from the player's own browser and nothing else. + this.socket = new ServerSocket(0, 1, InetAddress.getByName("127.0.0.1")); + } + + /** What to hand Google as {@code redirect_uri}. It must match at token exchange, byte for byte. */ + String redirectUri() { + return "http://127.0.0.1:" + socket.getLocalPort(); + } + + /** + * Blocks until the browser arrives with a query string, and answers it with + * a page telling the player to go back to the game. + * + *

Browsers open speculative connections and ask for {@code favicon.ico} + * unprompted, so anything without a {@code code} or {@code error} is turned + * away and the wait continues until the deadline. + */ + Map awaitRedirect(Duration timeout) throws IOException { + long deadline = System.currentTimeMillis() + timeout.toMillis(); + while (true) { + long remaining = deadline - System.currentTimeMillis(); + if (remaining <= 0) { + throw new SocketTimeoutException("Nothing came back from the browser"); + } + socket.setSoTimeout((int) Math.min(remaining, Integer.MAX_VALUE)); + try (Socket client = socket.accept()) { + client.setSoTimeout(READ_TIMEOUT_MILLIS); + Map parameters = read(client); + if (parameters.containsKey("code") || parameters.containsKey("error")) { + respond(client, parameters.containsKey("code")); + return parameters; + } + respond(client, false); + } catch (SocketTimeoutException e) { + throw e; + } catch (IOException e) { + // One browser connection going wrong is not the sign-in going + // wrong; the real one may still be on its way. + log.debug("Ignoring a loopback connection that failed: {}", e.toString()); + } + } + } + + private Map read(Socket client) throws IOException { + BufferedReader reader = new BufferedReader( + new InputStreamReader(client.getInputStream(), StandardCharsets.ISO_8859_1)); + String requestLine = reader.readLine(); + if (requestLine == null) { + return Map.of(); + } + String[] parts = requestLine.split(" "); + int query = parts.length < 2 ? -1 : parts[1].indexOf('?'); + return query < 0 ? Map.of() : parse(parts[1].substring(query + 1)); + } + + private static Map parse(String query) { + Map parameters = new LinkedHashMap<>(); + for (String pair : query.split("&")) { + if (pair.isEmpty()) { + continue; + } + int equals = pair.indexOf('='); + String name = equals < 0 ? pair : pair.substring(0, equals); + String value = equals < 0 ? "" : pair.substring(equals + 1); + parameters.put(decode(name), decode(value)); + } + return parameters; + } + + private static String decode(String value) { + try { + return URLDecoder.decode(value, StandardCharsets.UTF_8); + } catch (IllegalArgumentException e) { + return value; + } + } + + /** + * The page the player is left looking at. Deliberately plain text in a + * minimal document: it has to render from a socket that is about to close, + * with no stylesheet and no second request. + */ + private static void respond(Socket client, boolean signedIn) throws IOException { + String message = signedIn + ? "PhotoSync is signed in. You can close this tab and go back to Minecraft." + : "PhotoSync is still waiting for Google's answer. You can close this tab."; + byte[] page = ("PhotoSync" + + "

" + message + "

") + .getBytes(StandardCharsets.UTF_8); + OutputStream out = client.getOutputStream(); + out.write(("HTTP/1.1 200 OK\r\n" + + "Content-Type: text/html; charset=utf-8\r\n" + + "Content-Length: " + page.length + "\r\n" + + "Connection: close\r\n\r\n").getBytes(StandardCharsets.US_ASCII)); + out.write(page); + out.flush(); + } + + @Override + public void close() { + try { + socket.close(); + } catch (IOException e) { + log.debug("Closing the sign-in socket threw", e); + } + } +} diff --git a/shared/core/src/main/java/dev/photosync/core/provider/google/SignInPrompt.java b/shared/core/src/main/java/dev/photosync/core/provider/google/SignInPrompt.java new file mode 100644 index 0000000..c37d6da --- /dev/null +++ b/shared/core/src/main/java/dev/photosync/core/provider/google/SignInPrompt.java @@ -0,0 +1,31 @@ +package dev.photosync.core.provider.google; + +import org.slf4j.LoggerFactory; + +/** + * How the player is put in front of Google's consent page. + * + *

Google Photos has no API key to paste: the only way to get a token is to + * send the player to Google in a browser and catch what comes back. Core cannot + * open a browser -- that is a Minecraft capability -- so the client module hands + * one of these in, and this package stays free of the game. + */ +@FunctionalInterface +public interface SignInPrompt { + + /** + * Shows {@code url} to the player, ideally by opening their browser. + * + *

Called from a worker thread, once per sign-in. It does not have to + * succeed: the URL is logged as well, and the sign-in waits for the + * redirect either way, so a player whose browser will not open can still + * paste the link themselves. + */ + void show(String url); + + /** The fallback when nobody supplied one: the log is the only way in. */ + static SignInPrompt logging() { + return url -> LoggerFactory.getLogger(SignInPrompt.class) + .info("Open this page to finish signing in to Google Photos: {}", url); + } +} diff --git a/shared/mc-api/src/main/java/dev/photosync/mcapi/GameContext.java b/shared/mc-api/src/main/java/dev/photosync/mcapi/GameContext.java index 77bc6c7..90baee0 100644 --- a/shared/mc-api/src/main/java/dev/photosync/mcapi/GameContext.java +++ b/shared/mc-api/src/main/java/dev/photosync/mcapi/GameContext.java @@ -61,6 +61,22 @@ public interface GameContext { */ void reveal(Path path); + /** + * Opens a URL in the player's browser. + * + *

Here for the Google Photos sign-in, which cannot happen inside the + * game: the consent page is Google's, it is where the password is typed, + * and re-hosting it in a Minecraft screen would be both impossible and the + * wrong thing to teach a player to trust. + * + *

It crosses this bridge rather than calling {@code java.awt.Desktop} + * directly because AWT and GLFW cannot both own the main thread on macOS, + * where the game is launched with {@code -XstartOnFirstThread}. Minecraft + * ships an opener that already accounts for that, and it is the same one + * behind every link in the game's own menus. + */ + void openUrl(String url); + /** The running Minecraft version, for logs and the settings screen's footer. */ String minecraftVersion(); diff --git a/shared/ui/src/main/java/dev/photosync/ui/image/ThumbnailCache.java b/shared/ui/src/main/java/dev/photosync/ui/image/ThumbnailCache.java index 829fbfe..22a384d 100644 --- a/shared/ui/src/main/java/dev/photosync/ui/image/ThumbnailCache.java +++ b/shared/ui/src/main/java/dev/photosync/ui/image/ThumbnailCache.java @@ -37,6 +37,26 @@ import java.util.concurrent.CompletableFuture; @Slf4j public final class ThumbnailCache implements AutoCloseable { + /** + * How many times one asset's thumbnail is fetched before the tile settles + * for its blur. + * + *

The first attempt fails for all the ordinary reasons -- the server was + * restarting, the connection dropped, the loader handed back a truncated + * body -- and giving up on it permanently leaves a tile that is blurred for + * the rest of the session with nothing to say why. Three attempts covers + * the transient cases; past that the failure is real and is drawn as one. + */ + private static final int MAX_ATTEMPTS = 3; + + /** + * Frames to wait before trying again, multiplied by the attempt number. + * Long enough that a server that is down is not hammered by every tile on + * screen, short enough that the grid repairs itself while the player is + * still looking at it. + */ + private static final int RETRY_FRAMES = 40; + /** What to draw for one asset, and whether it is the real thing yet. */ public record Thumbnail(TextureHandle texture, boolean placeholder) { } @@ -89,10 +109,25 @@ public final class ThumbnailCache implements AutoCloseable { return entry.thumbnail(); } - /** Whether this asset's thumbnail failed outright, so the grid can mark it. */ + /** + * Starts an asset's fetch without drawing it, for a tile that is about to + * scroll into view. + * + *

Refuses once the cache is full, so lookahead can never evict a tile the + * player is actually looking at: the visible tiles are asked for first, and + * whatever room is left over is what the band gets. + */ + public void prefetch(RemoteAsset asset) { + if (!entries.containsKey(asset.id()) && entries.size() >= capacity) { + return; + } + of(asset); + } + + /** Whether this asset's thumbnail failed for good, so the grid can mark it. */ public boolean failed(String assetId) { Entry entry = entries.get(assetId); - return entry != null && entry.failed; + return entry != null && entry.exhausted(); } /** Call at the end of a frame, once every visible tile has been asked for. */ @@ -132,7 +167,8 @@ public final class ThumbnailCache implements AutoCloseable { private TextureHandle texture; private boolean real; - private boolean failed; + private int attempts; + private long retryFrame; private CompletableFuture pending; private long touched; @@ -145,12 +181,17 @@ public final class ThumbnailCache implements AutoCloseable { return texture == null ? Optional.empty() : Optional.of(new Thumbnail(texture, !real)); } + /** Out of attempts: whatever is drawn now is what this tile gets. */ + private boolean exhausted() { + return !real && attempts >= MAX_ATTEMPTS; + } + /** * Advances this entry by whatever is available without blocking: start a * request, or take delivery of one. */ private void poll() { - if (real || failed) { + if (real || exhausted() || frame < retryFrame) { return; } if (pending == null) { @@ -168,8 +209,10 @@ public final class ThumbnailCache implements AutoCloseable { adopt(textures.decode(finished.join())); real = true; } catch (IOException | RuntimeException e) { - log.debug("Thumbnail {} is not drawable: {}", asset.id(), e.toString()); - failed = true; + attempts++; + retryFrame = frame + (long) RETRY_FRAMES * attempts; + log.debug("Thumbnail {} is not drawable (attempt {} of {}): {}", + asset.id(), attempts, MAX_ATTEMPTS, e.toString()); } } diff --git a/shared/ui/src/main/java/dev/photosync/ui/screen/QueueScreen.java b/shared/ui/src/main/java/dev/photosync/ui/screen/QueueScreen.java index dab7db6..edd7e96 100644 --- a/shared/ui/src/main/java/dev/photosync/ui/screen/QueueScreen.java +++ b/shared/ui/src/main/java/dev/photosync/ui/screen/QueueScreen.java @@ -42,6 +42,23 @@ public final class QueueScreen extends PhotoSyncScreen { private static final int DETAIL_WIDTH = 140; private static final int MIN_WIDTH_FOR_DETAIL = 340; + /** + * How many times a screenshot is read off disk before the pane gives up. + * + *

One attempt is not enough, and the reason is specific to what this list + * holds. A row can appear the instant a capture is queued, while the PNG + * behind it is still being flushed by the IO pool; the screenshots folder is + * on a network drive or a spinning disk often enough; and an antivirus that + * has the file open will refuse one read and allow the next. Failing those + * permanently leaves the newest screenshot -- the one at the top of the + * list, the one the player actually clicks -- showing "unavailable" for the + * rest of the session, with no way to ask again short of changing tabs. + */ + private static final int PREVIEW_ATTEMPTS = 3; + + /** Milliseconds before the next attempt, multiplied by the attempt number. */ + private static final long PREVIEW_RETRY_MILLIS = 600; + private final UploadQueue queue; private final ScrollModel scroll; private final Preview preview = new Preview(); @@ -223,7 +240,7 @@ public final class QueueScreen extends PhotoSyncScreen { return; } UploadJob job = selection.get().job(); - preview.follow(job.path()); + preview.follow(job.path(), System.currentTimeMillis()); Rect inner = detailArea.inset(4); int line = render.lineHeight() + 2; @@ -366,20 +383,38 @@ public final class QueueScreen extends PhotoSyncScreen { private Path source; private CompletableFuture reading; private TextureHandle texture; - private boolean failed; + private int attempts; + private long retryAtMillis; /** Called every frame with the selected file; only acts when it changes. */ - private void follow(Path path) { + private void follow(Path path, long now) { if (!path.equals(source)) { close(); source = path; - reading = CompletableFuture.supplyAsync(() -> readAll(path)); } - poll(); + poll(now); } - private void poll() { - if (reading == null || !reading.isDone()) { + /** + * Advances by whatever is available without blocking: start a read, or + * take delivery of one. + * + *

The read is started here rather than in {@link #follow} so that a + * retry needs no separate path -- an attempt that failed simply leaves + * nothing pending, and the next frame past the backoff starts another. + */ + private void poll(long now) { + if (texture != null || source == null) { + return; + } + if (reading == null) { + if (attempts < PREVIEW_ATTEMPTS && now >= retryAtMillis) { + Path path = source; + reading = CompletableFuture.supplyAsync(() -> readAll(path)); + } + return; + } + if (!reading.isDone()) { return; } CompletableFuture finished = reading; @@ -387,8 +422,10 @@ public final class QueueScreen extends PhotoSyncScreen { try { texture = ui.bridge().textures().decode(finished.join()); } catch (IOException | RuntimeException e) { - log.debug("Cannot preview {}", source, e); - failed = true; + attempts++; + retryAtMillis = now + PREVIEW_RETRY_MILLIS * attempts; + log.debug("Cannot preview {} (attempt {} of {}): {}", + source, attempts, PREVIEW_ATTEMPTS, e.toString()); } } @@ -396,8 +433,9 @@ public final class QueueScreen extends PhotoSyncScreen { return Optional.ofNullable(texture); } + /** Out of attempts: this screenshot is not going to draw. */ private boolean failed() { - return failed; + return texture == null && attempts >= PREVIEW_ATTEMPTS; } /** Forgets the current image so the next {@link #follow} reloads. */ @@ -416,12 +454,22 @@ public final class QueueScreen extends PhotoSyncScreen { texture.close(); texture = null; } - failed = false; + attempts = 0; + retryAtMillis = 0; } private byte[] readAll(Path path) { try { - return Files.readAllBytes(path); + byte[] bytes = Files.readAllBytes(path); + if (bytes.length == 0) { + // The name is claimed by creating the file empty and the + // pixels follow on the IO pool, so a screenshot taken a + // moment ago is briefly a real file with nothing in it. + // Treating that as a decode failure would spend an attempt + // saying "not an image" about a file that is about to be one. + throw new CompletionException(new IOException(path + " has not been written yet")); + } + return bytes; } catch (IOException e) { throw new CompletionException(e); } diff --git a/shared/ui/src/main/java/dev/photosync/ui/screen/SettingsScreen.java b/shared/ui/src/main/java/dev/photosync/ui/screen/SettingsScreen.java index ed6cfe9..a7eb1e8 100644 --- a/shared/ui/src/main/java/dev/photosync/ui/screen/SettingsScreen.java +++ b/shared/ui/src/main/java/dev/photosync/ui/screen/SettingsScreen.java @@ -27,6 +27,7 @@ import java.util.ArrayList; import java.util.List; import java.util.Locale; import java.util.concurrent.CompletableFuture; +import java.util.concurrent.Executor; import java.util.function.BooleanSupplier; import java.util.function.Consumer; import java.util.function.UnaryOperator; @@ -48,6 +49,26 @@ import java.util.function.UnaryOperator; */ public final class SettingsScreen extends PhotoSyncScreen { + /** + * A thread of its own for "test connection", rather than the common pool. + * + *

The common pool is sized from the core count and is shared with every + * other {@code supplyAsync} in the mod -- the queue screen's previews among + * them. A connection test used to be a request with a timeout on it, which + * that pool could absorb; signing in to Google Photos is a wait for a human + * to find a browser window and press Allow, which it cannot. On a two-core + * machine one such wait is the whole pool. + * + *

Daemon, so a sign-in nobody ever finished does not keep the game from + * closing. The button is disabled while a test runs, so this makes at most + * one thread at a time. + */ + private static final Executor TEST_EXECUTOR = task -> { + Thread thread = new Thread(task, "photosync-connection-test"); + thread.setDaemon(true); + thread.start(); + }; + /** One line of the form. A null widget makes it a section heading. */ private static final class Row { @@ -358,7 +379,7 @@ public final class SettingsScreen extends PhotoSyncScreen { } catch (RuntimeException e) { testStatus = chrome.translate("photosync.settings.test.failed", e.toString()); } - }).whenComplete((ignored, failure) -> ui.game().submit(() -> test.enabled(true))); + }, TEST_EXECUTOR).whenComplete((ignored, failure) -> ui.game().submit(() -> test.enabled(true))); } // ----------------------------------------------------------------------- diff --git a/shared/ui/src/main/java/dev/photosync/ui/screen/TimelineScreen.java b/shared/ui/src/main/java/dev/photosync/ui/screen/TimelineScreen.java index 950fabe..e08a11f 100644 --- a/shared/ui/src/main/java/dev/photosync/ui/screen/TimelineScreen.java +++ b/shared/ui/src/main/java/dev/photosync/ui/screen/TimelineScreen.java @@ -167,6 +167,10 @@ public final class TimelineScreen extends PhotoSyncScreen { scroll.advance(System.currentTimeMillis()); chrome.well(render, grid); + // The tile cache's frame spans the whole body rather than just the grid, + // because the opened photo borrows a tile texture and would otherwise be + // asking for one that this frame's eviction pass had already dropped. + tiles.beginFrame(); if (renderState(render)) { renderGrid(render, mouseX, mouseY); scroll.render(render, mouseX, mouseY); @@ -174,6 +178,7 @@ public final class TimelineScreen extends PhotoSyncScreen { if (opened != null) { renderOpened(render, mouseX, mouseY); } + tiles.endFrame(); } /** Draws whatever stands in for the grid, and says whether the grid itself should be drawn. */ @@ -210,15 +215,15 @@ public final class TimelineScreen extends PhotoSyncScreen { } private void renderGrid(RenderBridge render, int mouseX, int mouseY) { - tiles.beginFrame(); int offset = scroll.offset(); // One viewport of lookahead, so a month is already being fetched by the // time the player scrolls it into view. int prefetchTop = offset - grid.height(); int prefetchBottom = offset + grid.height() * 2; + int first = firstVisible(prefetchTop); render.pushClip(grid.x(), grid.y(), grid.width(), grid.height()); - for (int i = firstVisible(prefetchTop); i < blocks.size(); i++) { + for (int i = first; i < blocks.size(); i++) { Block block = blocks.get(i); if (block.top() > prefetchBottom) { break; @@ -232,7 +237,45 @@ public final class TimelineScreen extends PhotoSyncScreen { } } render.popClip(); - tiles.endFrame(); + + // Second pass on purpose: every visible tile has now had its turn at the + // loader, so the band gets whatever request slots and cache room are + // left rather than competing for them. + for (int i = first; i < blocks.size(); i++) { + Block block = blocks.get(i); + if (block.top() > prefetchBottom) { + break; + } + prefetchBlock(block, prefetchTop, prefetchBottom); + } + } + + /** + * Asks for the thumbnails of the tiles between {@code top} and {@code + * bottom} in content space, without drawing them. + * + *

Fetching a thumbnail only once its tile is inside the viewport means + * every tile is blurred for a whole round trip after it appears, which over + * a scroll is most of what the player sees. The months were already being + * loaded a viewport ahead; this gives their images the same head start. + */ + private void prefetchBlock(Block block, int top, int bottom) { + List assets = block.section().assets(); + if (assets.isEmpty()) { + return; + } + int stride = block.tileSize() + theme().tileGap(); + int gridTop = block.gridTop(headerHeight); + int lastRow = (bottom - gridTop) / stride; + if (lastRow < 0) { + return; + } + int firstRow = Math.max(0, (top - gridTop) / stride); + int from = firstRow * block.columns(); + int to = Math.min(assets.size(), (lastRow + 1) * block.columns()); + for (int index = from; index < to; index++) { + tiles.prefetch(assets.get(index)); + } } private void renderBlock(RenderBridge render, Block block, int offset, int mouseX, int mouseY) { @@ -278,7 +321,10 @@ public final class TimelineScreen extends PhotoSyncScreen { render.fill(tile.x(), tile.y(), tile.width(), tile.height(), theme().tilePlaceholder()); Optional thumbnail = tiles.of(asset); thumbnail.ifPresent(value -> drawCropped(render, value.texture(), tile)); - if (thumbnail.isEmpty() && tiles.failed(asset.id())) { + // Marked even when there is a ThumbHash to draw. A tile that keeps its + // blur because the image never arrived looks exactly like one the mod + // simply chose not to sharpen, and the player has no way to tell. + if (tiles.failed(asset.id())) { chrome.centered(render, "!", tile, theme().danger()); } if (asset.isVideo() && ui.core().config().current().browser().showVideoBadge()) { @@ -302,33 +348,49 @@ public final class TimelineScreen extends PhotoSyncScreen { /** * The opened photo, at whatever resolution has arrived so far. * - *

What arrives first is the ThumbHash: a blurred 32x32 that is right for a - * tile and nowhere near enough for a full-screen view. It is still worth - * drawing, because it says which photo is opening, but it is dimmed and - * labelled while it stands in -- an unannotated blur is indistinguishable - * from a mod that fetched the wrong size. + *

The stand-in while the full image loads is the grid's own thumbnail: + * it is already on the GPU, it is the picture the player just clicked, and + * it is sharp enough to be worth looking at. Only when the grid has nothing + * real either does this fall back to the ThumbHash -- a blurred 32x32 that + * is right for a tile and nowhere near enough for a full-screen view, so it + * is dimmed and labelled while it stands in. An unannotated blur is + * indistinguishable from a mod that fetched the wrong size. */ private void renderOpened(RenderBridge render, int mouseX, int mouseY) { detail.beginFrame(); Rect area = body(); render.fill(area.x(), area.y(), area.width(), area.height(), theme().overlay()); Rect frame = area.inset(6); - Optional image = detail.of(opened); + Optional full = detail.of(opened); + // Keeps asking the grid cache too, so the tile survives this frame's + // eviction pass even when the player has scrolled it out of the grid. + Optional tile = tiles.of(opened); - if (detail.failed(opened.id())) { + if (full.isPresent() && !full.get().placeholder()) { + drawContained(render, full.get().texture(), frame); + } else if (detail.failed(opened.id())) { chrome.notice(render, frame, chrome.translate("photosync.browse.open_failed"), chrome.translate("photosync.browse.open_failed.hint")); - } else if (image.isEmpty()) { - chrome.notice(render, frame, chrome.translate("photosync.browse.opening"), ""); } else { - drawContained(render, image.get().texture(), frame); - if (image.get().placeholder()) { - render.fill(frame.x(), frame.y(), frame.width(), frame.height(), - theme().fade(theme().overlay(), 0.6f)); - chrome.centered(render, chrome.translate("photosync.browse.opening"), - new Rect(frame.x(), frame.centerY() - render.lineHeight(), frame.width(), render.lineHeight()), - theme().text()); - chrome.busyBar(render, new Rect(frame.centerX() - 60, frame.centerY() + 6, 120, 3), + Optional standIn = + tile.filter(image -> !image.placeholder()).or(() -> full); + standIn.ifPresent(image -> { + drawContained(render, image.texture(), frame); + if (image.placeholder()) { + render.fill(frame.x(), frame.y(), frame.width(), frame.height(), + theme().fade(theme().overlay(), 0.6f)); + chrome.centered(render, chrome.translate("photosync.browse.opening"), + new Rect(frame.x(), frame.centerY() - render.lineHeight(), + frame.width(), render.lineHeight()), + theme().text()); + } + }); + if (standIn.isEmpty()) { + chrome.notice(render, frame, chrome.translate("photosync.browse.opening"), ""); + } else { + // Under the picture rather than over it, because the picture is + // now worth seeing: the bar says a sharper one is still coming. + chrome.busyBar(render, new Rect(frame.centerX() - 60, frame.bottom() - 6, 120, 3), System.currentTimeMillis(), theme().accent()); } }