[Upgrade Ubuntu] be_prod

[dacook]

Slack thread: https://openfoodnetwork.slack.com/archives/CG7NJ966B/p1732147234656099

• Check old server config for any additional services to be aware of. Document any necessary steps for migration.
• take snapshot in case of failure
• set up admin user in case need to investigate

migration

• in-place upgrade to Ubuntu 18
  • sudo systemctl stop nginx puma sidekiq redis* postgres
  • sudo do-release-upgrade, press Y lots of times..
• in-place upgrade to Ubuntu 20
• test reboot
• ensure playbooks work
  • provision.yml
  • deploy.yml

Testing

• send test mail (/admin/mail_methods/edit).
• terms of service file: /admin/terms_of_service_files
• shop catalogue display correctly, with images, add to cart, begin checkout, login
• note: check cookies if login won't work
• Check integrations
  • Payments (check Stripe connect status /admin/stripe_connect_settings/edit)
  • New Relic
  • Bugsnag
• Check there are no alarm bells, eg:
  • tiny data change is present. undo it.
  • upload a product image
  • ~/apps/openfoodnetwork/current/log/production.log and sidekiq.log

follow up

• Update documentation:
  • https://github.com/openfoodfoundation/ofn-install/wiki/Current-servers

Rollback plan

• If unable to resolve any issues in-place, restore snapshot
• If unable to restore snapshot....contact OVH suport
• If no response... seek contact with Emmanuel.. provision server on new hosting provider with own credit card?

[dacook]

Vincent has left OMdM and I have had this email from his manager - maybe you could contact him:

On Fri, 22 Nov 2024 08:20:45 +0000 Emmanuel Bawin <[email protected]> wrote
Dear Nick,
Thanks for you message.
Indeed Vincent do not work anymore for us.
I will meet our different Belgian partners soon to clarify the repartition of roles.
Until this, I will be the point of contact.

[dacook]

Current stats:

htop
  1  [                                              0.0%]   Tasks: 55, 138 thr; 1 running
  2  [|                                             0.7%]   Load average: 0.08 0.10 0.09
  3  [|                                             1.3%]   Uptime: 563 days(!), 13:46:43
  4  [                                              0.0%]
  Mem[|||||||||||||||||||||||||||||||||||||||2.90G/7.61G]
  Swp[                                             0K/0K]

df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            3.8G     0  3.8G   0% /dev
tmpfs           780M   81M  699M  11% /run
/dev/sda1        97G   30G   67G  31% /

[dacook]

Sigh I've just realised I need to take a snapshot before starting, because I don't have any other rollback plan.
This requires an extra payment on the account, which requires the card security code.

Pausing and will proceed once snapshots are enabled.

[dacook]

Took about an hour, with about 15 mins unexpected downtime, but I now have a snapshot, which presumably I can restore in the case of an unresolvable issue.

[dacook]

Unfortunately the in-place upgrade won’t work (without a lot of manual hacking), so we need to provision a new server. I have found a local provider (is*hosting) and will attempt to set up as much as I can until Emmanuel can provide payment.

For reference:

On Thu, 12 Dec 2024 at 16:25, David Cook [email protected] wrote:

Hi Emmanuel,

I'm sorry to have to come back to you without success. Although in similar situations it had worked, I wasn't able to successfully upgrade your server in place, so we will need to choose a more reliable alternative.

From our previous communication, I guess that you'd prefer a server located close to Belgium. I have reviewed a few other options and will share my recommendation.

• Combell: expensive at 84.99/month (4vcpu 8gbram)
• Easyhost eur24/month (4vcpu 8gbram) Some bad reviews: https://au.trustpilot.com/review/easyhost.be?languages=all
• is*hosting $30/month (approx eur29) in belgium (3vcpu 8gbram) Reviews look fine: https://au.trustpilot.com/review/ishosting.com
• Infomaniak in switzerland eur29/month (4vcpu, 12gbram). Recommended by other OFN users

is*hosting seemed like a good option, so I have created an account and added the required VPS to the cart. It's ready to checkout, so if you are happy with this, can you please log in and add credit card details?
The login details are:
https://my.ishosting.com/en/cart
[email protected]
RndZ&z3U&6S8Cfv

If you would prefer to create the account yourself, or choose another hosting provider, note the key details needed for a server:
VPS
3+ CPU
8+ GB RAM
40+ GB storage
Ubuntu 20.04 (this is important)

[dacook]

Backup failed at 10am. I'm not sure if it continued to fail, need to login to check.
https://openfoodnetwork.slack.com/archives/CEF14NU3V/p1733959042938539

---

Logged in, it was just a single network error, all fine now.