Building Dropshare Cloud
Last year I founded chaos.cloud UG (haftungsbeschränkt*) together with Timo. Timo is known for building Dropshare, a macOS and iOS native file sharing app. Dropshare allows users to upload files by dragging and dropping to the menubar as well as screenshots, screenrecordings and capturing Android and iOS screenshots via USB. Dropshare Cloud is one of the supported storage options, built and operated by us. We try to offer the best user experience as well as advanced features like statistics, auto expiry for files, custom domain as well as SSL/TLS for everyone.
AWS S3 was the obvious option for storing files. It's highly reliable, reasonable priced and equipped with awesome APIs. We started out under the assumption that we could create a bucket per user and have S3 deliver the files. This quickly turned out to be a wrong assumption as creating more than 100 buckets is not easily possible. Luckily there is a workable solution: Creating temporary credentials via AWS Security Token Service (STS). The service allows creating temporary, limited-privilege login credentials for AWS services. It's configurable via the normal AWS IAM policies, so it's possible to create credentials that are only allowed to access parts of bucket by restricting the access rights to a prefix of the bucket. We operate a webservice that allows the app to request access key and secret for a users path on S3 via HTTPS. The credentials have a limited time of validity and are restricted to only operate on the users prefix on S3.
Delivery and statistics
Storing files is done, next part is delivering them for people to see. We proxy all files for several reasons: – We need to control the URL for the looks – We want to offer SSL – We want to offer custom domains (with free SSL certificates) – We need quick and granular billing To proxy the files we operate caching proxies. They are based on a custom stack built around OpenResty and Varnish, as well as custom statistics collection software.
We use OpenResty for flexible proxying as well as dynamic serving of SSL certificates. Requests for our shared domain are more or less passed to varnish and S3 directly, for custom domains we resolve the domain to the users prefix on S3 and then hand off to varnish. SSL is included in all our plans, the “Power” level also offers free SSL certificates on a users custom domain. To offer this service we leverage LetsEncrypt and dynamically request certificates for custom domains. OpenResty offers an ingenious way of loading SSL certificates dynamically during the SNI handshake which we use to fetch them from a local Redis readslave.
Once we started serving files our own it was a wise choice to implement some form of caching. Varnish also renders custom error pages on S3 and has a little VCL logic to determine acceptable caching times. To be fully encrypted we take advantage of Stunnel, which we use to build a transparent TLS tunnel to S3, which Varnish itself doesn't support.
Our plans are limited on storage and transfer. We retrieve storage stats from S3, traffic stats are generated on our side. A worker is constantly following the Varnish NCSA log, parsing it for the bytes transferred in the client body. This information is put in a few different sorted sets in Redis for tracking the traffic of every file of a a user, also broken down per month. Those stats are interpreted by our main Ruby on Rails app in workers that check the usage stats regularly, disabling file delivery if limits are overstepped.
We decided we need to show a list of all files that have been uploaded for a user. The first naive implementation of this feature was just doing the S3 list call for the users prefix as the file browser was opened. This proved unbearably slow during the first extended tests with real amounts of uploads. Adding caching helped the situation for a while but meant we ran into all sorts of issues (users deleting a file or changing something meant our web UI was way off most of the time).
To solve this once and for all we resorted to AWS Lambda processing S3 triggers. A simple lambda function is called on each add/delete to S3. This function immediately messages us with the details of the change. We then update our internal files database with all information that is needed for displaying the file list. Switching from our cached list to this allowed file uploads to show almost immediately while reducing the complexity of our logic immensely. We currently have a worker running every night to make absolutely sure each and every file is found in our database, it yet has to find a difference.
This feature proved well worth the effort while implementing automatic expiry, as expired files are instantly reflected in the UI. Automatic expiry
To deal with the storage usage we offer automatic expiry of files. A user can select files based on extension or them being screenshots for setting an expiry on predetermined levels between one day and year. The selection is transferred to the app which sets S3 tags on the files. Using S3's auto expiry feature we set up rules that expire those files the set amount of time after the upload. This relatively new options frees us from iterating over all files and implementing such a logic ourselves, reducing the expenses for all S3 calls and worker capacity on our servers.
Finding the right payment solution was a more complex task than imagined. We started under the assumption that just going with Stripe would do the trick. We then decided that the support of PayPal would be desirable (we are in Germany, sadly having a credit card is far from normal here). After a quick trip to BrainTree land we switched to Paddle. They offer handling all the VAT stuff, processing PayPal and credit cards - so rewrite number three of the payment system is what happened, which since worked great. Paddle is less well supported than Stripe or Braintree, but implementing a few webhooks and a few models didn't take too long at all.
Be sure to check us out on Stackshare, try Dropshare Cloud and ask away in the comment section below. Also make sure to subscribe to Timo and me for more details about building our own product. *yep, it's the law to put that there…thanks Merkel