I'm a little curious why AWS is free/cheap, but anything other than AWS ends up in the "Enterprise, contact us for pricing" bucket. It might simply be based on your costs. I don't know if the on-prem stuff is expensive since your software needs to be in the same datacenter or if it's just a pricing differentiator.

I'm also wondering if you've thought of going the DBaaS route. It looks like AWS is charging an ~80% premium for RDS over regular on-demand instances. If you're managing a fleet of them and getting lower pricing (through committed-use discounts and negotiation) you could potentially get good margin there. For example, AWS charges $280/mo (on-demand) for an M5.2xLarge and $500/mo (on-demand) for the same instance for MySQL RDS (single AZ). So Amazon is already asking for an additional $220 and then you'd like $90/database ($450 / 5 databases on the standard plan). It seems like there might be more opportunity in the DBaaS markup than in trying to convince someone to use your tuning engine.

From your video, it doesn't sound like OtterTune does this, but it might be worth thinking about: maybe not just tuning the database, but also offering suggestions on potential anti-patterns happening. For example, if someone has code that should be using connection pools and isn't, it could be helpful to present it as something potentially important. Likewise, let's say that there are unindexed queries. Detecting that and recommending solutions is helpful. That seems like it could offer ongoing benefit.

Speaking of ongoing benefit, would there be a lot of benefit to paying for more than one month? In the 5 minute video, it showed the configuration stabilizing pretty quickly once it reached a good place. Assuming that my access patterns remain relatively the same, it seems like I could just continue with that configuration. Maybe I could sign up for one month every year to get a check-up. This isn't meant to be a criticism. It's more that I've seen so many customers exploit holes in SaaS pricing. If you're counting on recurring revenue, what happens if a customer notices that after a day they don't need the knobs tuned anymore? Can I get that $450 5-DB plan and run it on 5 databases on Monday, remove it from those 5 databases on Tuesday and apply it to 5 new databases on Tuesday (effectively getting 150 databases per month for the price of 5)? Or maybe there is a need to be tuning the configuration hourly/daily (though that seems less likely).

I do like the idea of auto-tuned knobs and it seems like a great service. I guess it's just hard to try if, like me, you're not on AWS. I'm sure that most people are so I'm not suggesting it's an inappropriate restriction for the free tier. Really cool and congrats on VLDB!

> I'm a little curious why AWS is free/cheap, but anything other than AWS ends up in the "Enterprise, contact us for pricing" bucket. It might simply be based on your costs. I don't know if the on-prem stuff is expensive since your software needs to be in the same datacenter or if it's just a pricing differentiator.

The expensive part for us about on-prem is the amount of custom code that we have to write for the agent to deal with how the organization maintains their config files. For example, one customer that we talked maintained TerraForm files for their MySQL confs in Github. So we had to modify the Github repo, push to main, that would then trigger a Github action that then pushed the update to database. Then we had to know when the updated config was deployed so that we could bounce the system (if it required a restart). Restarting the DBMS is tricky too, since everybody does it differently.

> I'm also wondering if you've thought of going the DBaaS route.

There is a lot to running a DBaaS (backups, configs, upgrades). At this point I think it would be very difficult to compete with Amazon/MSFT/Google/Oracle/etc if it was just stock Postgres/MySQL. You need a killer feature

> Speaking of ongoing benefit, would there be a lot of benefit to paying for more than one month?

You are asking about a floating license. This is a potential for enterprise customers. For now we track RDS databases by their ARN. So if you try to switch that on OtterTune, it treats it as a new database. This is necessary to make sure that the ML models don't freak out if there is a dramatic change all of a sudden.

The demo video was with TPC-C. The workload is stable. We've seen workloads that evolve a lot over the course of months, so it is unlikely that the same configuration would be optimal during this entire period.

> I guess it's just hard to try if, like me, you're not on AWS.

Reach out to us and I'm happy to talk to you about your setup: https://ottertune.com/contact

> Really cool and congrats on VLDB!

The science comes first.

See the RDS costing is not fully transparent. you pay CRAZY LOT for backups - backup to s3 is impossible, except in a weird parquet way. This is a cost lock-in.

you again pay CRAZY LOT for network egress. so i cant use a Vercel app with RDS. ill just spend a lot on network.

these two features and anybody will switch: 1. cheap network traffic 2. cheap backups.

charge me 20% above RDS (for the base machine) and ill pay happily.

A different take. Instead of coming up with one more DBaaS, they can just focus on making the existing DBs better and smarter that could potentially bring back a lot of those customers moved to cloud to back to on-prem.

Wild guess, but I'm assuming only AWS support is implemented right now but they want people to contact them to gauge interest.

As for the Postgres-derivates that you mentioned (Timescale, Citus), we have not tested OtterTune for them yet. But there is no reason they shouldn't also work because they expose the same metrics API (pg_stat_database) and config knobs. There are just way more people using Postgres (RDS, Aurora), so we are focusing on them right now.

First of all, congrats for the project.

As you probably know, you and any other user can indeed manage Postgres configurations, in any environment, via an API. See the "MANAGE" section in https://postgresqlco.nf/ Essentially, a hosted "Git repository" for Postgres configurations.

As I said above, everyone that we've talked to rolls their conf management tools because they are doing more than just DBMS confs (proxies, networking, kernel params, middleware, etc). Lots of Terraform.

Yes, indeed we have considered doing a TF module for the configuration management API :)

FWIU, Trove supports 10+ databases including MySQL and PostgreSQL.

Here's the official Ansible role for Trove, which provisions various e.g. SQL databases on an OpenStack cloud: https://github.com/openstack/openstack-ansible-os_trove

https://m.youtube.com/watch?v=4d95fnGrSy8

I remember it contains two use cases other than AWS RDS.

I'm thinking more specifically about indexes.

In my organization, we provide a solution where clients can run complex analytic queries periodically. We frequently have to tune the DB manually by adding an index here or there (or just brute force by switching to a bigger RDS...).

A product analyzing the query logs, combined with the estimated size of the tables plus some EXPLAINs run, and then using that to provide index/schema recommendations could help a lot.

edit: The original research paper: https://cs.cmu.edu/~ggordon/van-aken-etal-parameters.pdf

For most workloads, shared_buffers and work_mem have the most impact. For write-heavy OLTP workloads, tuning the WAL (max_wal_size) and autovacuum knobs (e.g., autovacuum_vacuum_scale_factor) have the most benefit. For read-mostly OLAP workloads, Postgres' parallel knobs (max_parallel_workers_per_gather) provide the most improvement.

But you need to also tune all the other knobs to get the last 15-40% of potential performance improvement. This is what OtterTune can do.

More info about how we select knobs: https://ottertune.com/blog/prevent-machine-learning-from-wre...

>The OtterTune Agent connects to the database to retrieve the necessary information. OtterTune only requires access to retrieve the database's runtime metrics and configuration information (knobs). It does need to access user tables or queries.

Presumably, the last line should say OtterTune does NOT need to access tables/queries?

I see you can setup using CloudFormation. Any chance terraform is on the roadmap?

On a personal note, thanks for everything you do. I’m a subscriber to the CMU Database Group channel and advocate of keeping it real.

Yes, we will be adding support for TerraForm by next month.

> On a personal note, thanks for everything you do.

You're welcome! But database research is probably keeping me out of jail. That's why I have to keep going.

> I’m a subscriber to the CMU Database Group channel and advocate of keeping it real.

Don't thank me. Thank the Steven Moy Foundation for Keeping it Real (https://stevenmoyfoundation.org).

If I may, can you please shed light on why Peloton had to be archived and in essence re-done with OtterTune. Interested in your team's learnings from it from a software engineering point of view.

Some additional Qs:

- How did the team ensure this project doesn't suffer from the same disadvantages as its predecessor?

- What would you advise other teams undertaking a rewrite to pay off their tech debts?

How does this project compare to / contrast with Google's and SingleStore's efforts in this space?

Any chance we see you do a Peter Bailis and Sisu Data this? (:

Thanks.

Peloton and OtterTune are completely different projects. Peloton was abandoned and rewritten as NoisePage (https://noise.page). OtterTune has always been OtterTune.

See this recent interview where I discuss why we gave up on Peloton:

https://www.ibm.com/cloud/blog/database-deep-dives-with-andy...

> - How did the team ensure this project doesn't suffer from the same disadvantages as its predecessor?

Again, different projects. OtterTune is all about not having to modify the internals of Postgres, MySQL, and any other DBMS. This is why we were able to support Oracle in the academic version in a short amount of time:

https://ottertune.com/blog/vldb-autonomous-database-tuning-i...

> - What would you advise other teams undertaking a rewrite to pay off their tech debts?

It is hard for to provide general advice for this question because every situation is different.

> How does this project compare to / contrast with Google's and SingleStore's efforts in this space?

I am not familiar with Google or SingleStore using ML in the manner that we are with OtterTune to tune configuration knobs. Or at least I have not seen anything public about it.

These days Oracle is the most aggressive with pushing automated tuning capabilities (Oracle's autonomous DBaaS, AutoPilot for MySQL Heatwave). The difference with these approaches and OtterTune is that right now we are focused on configuration tuning (to avoid data privacy issues) and our core approach is platform/DBMS agnostic.

> Any chance we see you do a Peter Bailis and Sisu Data this? (:

I don't know what you mean by this? Peter Bailis is the Ryan Gosling of databases.

I understand its complex for on-prem. Instead of you tweaking the product to be compatible for every other company why not let the companies to be compatible with your standard. I understand it's easier to be said than done. Especially in the early innings.

PS: I really want to try OtterTune but I don't have AWS RDS. I have on-prem (aka Raspberry Pi) and I will change everything the way you want to have OtterTune run on my Raspberry Pi.

But on a $450 plan a 1 user limit is just absurd. So what? me and my other fellow senior dev can't BOTH access your service for this amount of money when clearly the real limit you are paying for is the amount of databases you can monitor.

If you really want GCP, let us know here: https://ottertune.com/contact

But as you add new app features, grow database size, upgrade DBMS versions, and make other changes, OtterTune tracks your system to make sure you are always using the best config. This is problem is even harder with 1000s of individual database instances. Customers have told us that they want something can tune database continuously in a way that is not possible with DBAs and existing monitoring tools.

Knob tuning is also the first step in the kind of automation that we are working on using OtterTune's ML-based approach.

We are working on the ability for OtterTune to alert you during monitoring mode when it thinks your workload/database have changed enough that it merits turning the tuning mode back on. We could also have it show the recommendations when this occurs as you suggest.

My DB configuration is about the last thing I want changing while I sleep.

Isn’t the main use case for it for savings? In other words less expensive setups.

Optimizing MySQL/Postgres RDS in the free version is low risk.

We are incrementally expanding the AZs we support in the free tier in order to carefully scale our service. We plan to support EU zones by the end of the year.

Otters are also vicious animals, which is an important trait to have if you are working with databases: https://youtu.be/J7f6s2g8C0I

https://en.m.wikipedia.org/wiki/Cot%E2%80%93caught_merger https://en.m.wikipedia.org/wiki/Flapping