# Exploring the Asset Storage Architecture of Penpot **URL:** \ **Category:** Inside Penpot\ **Tags:** internals, architecture\ **Created:** [June 9, 2025, 11:48am UTC](https://community.penpot.app/t/exploring-the-asset-storage-architecture-of-penpot/8975 "2025-06-09T11:48:22Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![niwinz](https://dub1.discourse-cdn.com/flex017/user_avatar/community.penpot.app/niwinz/32/122_2.png) [@niwinz](https://community.penpot.app/u/niwinz)\ **Post date:** [June 9, 2025, 11:48am UTC](https://community.penpot.app/t/exploring-the-asset-storage-architecture-of-penpot/8975/1 "2025-06-09T11:48:22Z")
## Introduction The main objective of this article is to explain how the asset storage system in **Penpot** works, along with its internal processes. Before diving into implementation details, it’s important to understand why Penpot needs a dedicated asset storage system in the first place. Unlike applications that simply store user-uploaded assets and delete them when the corresponding object is deleted, **Penpot is an editor**. This means users can create elements that contain images, then undo and redo those changes. These scenarios require **decoupling the asset lifecycle from the editing process** , making it **asynchronous** and **eventually consistent**. Additionally, Penpot uses **logical deletion** for most objects, allowing users a window of a few days to undo destructive actions. This further supports the need for asset storage to be decoupled from the objects we manage. ### Key Features Key features of the asset storage system include: - Logical and/or deferred deletion - Asset categorization via buckets - Asset deduplication within buckets - Reference counting by bucket - Support for multiple storage backends * * * ## The Data Model Before diving into the features, we’ll provide an overview of the data model, focusing on the two most relevant objects. ### `storage_object` Every uploaded asset creates an entry in the `storage_object` table (within Penpot’s PostgreSQL database). This table stores basic metadata such as size, type, and the backend where the asset is stored. **Schema:** ```auto Table "public.storage_object" Column | Type | Collation | Nullable | Default ------------+--------------------------+-----------+----------+-------------------- id | uuid | | not null | uuid_generate_v4() created_at | timestamp with time zone | | not null | now() deleted_at | timestamp with time zone | | | size | bigint | | not null | 0 backend | text | | not null | metadata | jsonb | | | touched_at | timestamp with time zone | | | ``` **Notable aspects:** - **`backend`** : Indicates where the physical asset is stored. Current options are `fs` and `s3`. A now-deprecated backend stored files in the database itself, which helped with backups but didn’t scale well. - **`bucket`** (in `metadata`): Categorizes the asset semantically. Main buckets include `team-font-variant`, `file-media-object`, and `profile`. - **`hash`** (in `metadata`): A BLAKE2b hash computed from the asset’s content, used for deduplication (within buckets). - **`touched_at`** : Marks whether the object is pending reference analysis. A `NULL` value indicates no recent changes requiring reanalysis. ### `file_media_object` When a user uploads an image in the workspace, an entry is also created in the `file_media_object` table. This object is what Penpot files internally reference. **Schema:** ```auto Table "public.file_media_object" Column | Type | Collation | Nullable | Default --------------+--------------------------+-----------+----------+-------------------- id | uuid | | not null | uuid_generate_v4() created_at | timestamp with time zone | | not null | clock_timestamp() deleted_at | timestamp with time zone | | | name | text | | not null | width | integer | | not null | height | integer | | not null | mtype | text | | not null | file_id | uuid | | not null | is_local | boolean | | not null | false media_id | uuid | | not null | thumbnail_id | uuid | | | ``` This table links a file to a `storage_object` through the `media_id` and `thumbnail_id` fields. For instance, multiple `file_media_object` entries (e.g., from different templates) can reference the same `storage_object` thanks to deduplication. This also makes garbage collection (GC) more efficient, as it can query references directly without scanning file blobs. Other objects like fonts and profile photos reference `storage_object` directly along with their corresponding bucket names. * * * ## Processes ### Image Uploads When a user uploads an image (as-is or as a background), a multipart request is made to the API. And internally the upload process performs the following operations: 1. The BLAKE2b hash of the content is calculated. 2. The system checks for an existing `storage_object` with the same hash and bucket (per example `file-media-object`). 3. A new entry in `file_media_object` is created linking the asset and the file, storing metadata like size and MIME type. 4. The API returns the `file_media_object` ID. 5. The frontend updates the file with this ID. For fonts or other object types, there’s no intermediate relation; they reference the `storage_object` directly. If the image is no longer used (e.g., after an undo), it remains referenced until the file becomes **inactive** (i.e., no recent modifications) and is processed by **FileGC** (see below). ### Logical Deletes Although not exclusive to asset storage, logical deletion is tightly related. When a major object (file, project, team, profile) is deleted, it’s **marked as deleted** , and an asynchronous cascade process begins. This marks related objects as deleted with the same timestamp. Actual removal is deferred to the **Garbage Collection** process, offering a aprox 7-day undo window. * * * ## Garbage Collection We often joke that Penpot’s GC is like a **Mark and Sweep** garbage collector, with a marking phase (analysis) and a sweep phase (deletion). There are four main GC processes: ### **FileGC** Cleans up old or unused image references in a file after a period of inactivity. It also performs many other file cleaning operations and it consists in: - Analyzes file content. - Deletes unused entries from `file_media_object`. - Marks the corresponding `storage_object` entries via the `touched_at` field. Runs periodically, only processing inactive files. ### **ObjectsGC** Deletes all non-storage objects marked for deletion after ~7 days. If the object is linked to a `storage_object`, it sets the `touched_at` field for later analysis. ### **StorageTouchedGC** Periodically scans `storage_object` entries with a non-null `touched_at`. - Determines the reference strategy based on the object’s bucket. - If no references exist, marks the object as deleted for final removal. ### **StorageDeletedGC** Permanently deletes `storage_object` entries marked as deleted. Uses **batch deletion** to optimize throughput—particularly important for S3-type backends that may charge per API call. * * * ## Performance Considerations These processes must run **efficiently and incrementally** , avoiding long-held locks. A slow GC process can block user actions like uploading images or templates, leading to performance issues or timeouts. This is especially important with deduplication. If a GC process is analyzing a `storage_object` already used in a new template being uploaded, the upload may be blocked until GC completes. To mitigate this, each GC process uses **mini-transactions** , reducing locking time and making operations virtually invisible to users. ---
**Author:** ![Rayan\_Beigh](https://dub1.discourse-cdn.com/flex017/user_avatar/community.penpot.app/rayan_beigh/32/7351_2.png) [@Rayan\_Beigh](https://community.penpot.app/u/Rayan_Beigh)\ **Post date:** [July 2, 2025, 8:29pm UTC](https://community.penpot.app/t/exploring-the-asset-storage-architecture-of-penpot/8975/2 "2025-07-02T20:29:00Z")
Penpot is my favourite tool and a project that I want to contribute to. Posts like this are very helpful in understanding the overall project better. Keep posting regarding architecture and Penpot internals ❤