Today (15 Aug 2018) at around 7.10pm PST we received an alert that SYNCbits is not working properly. What followed was around 1 hour outage of service and I’d like to explain what happened and what we did to make sure this does not happen again.
A few minutes after finding out about the outage, we’ve stopped SYNCbits so we can investigate without affecting anyone’s data. Between the time the outage started though and the time we stopped it, some of you were logged out of SYNCbits.
After investigation we found out that the issue was caused because some of the SYNCbits servers that serve web traffic (not database servers) automatically updated which resulted in PHP updating to the latest version. Apparently one of the modules that we use has been removed from PHP 7.2 which we were unaware of until now. The module in question is mcrypt and it’s responsible for encrypting and decrypting your data.
When PHP was auto-updated to version 7.2 that module stopped working. The result was that every time MoneyWiz or MileWiz would try to sync, they’d transmit the password to authenticate you, but since your password failed to be decrypted (as the module responsible for that has been removed), SYNCbits returned error that the password is wrong and MoneyWiz/MileWiz logged you out.
Here is what we did and what we plan to further do to avoid this:
1. We manually downgraded PHP to version 7 to ensure compatibility with the needed module.
2. We further migrated all existing servers to that architecture
3. We also disabled automatic updates on these servers
4. SYNCbits is now restored but we’ll investigate what can we use instead of mcrypt so we can update to PHP 7.2 in the future without any downtime for you.
What should you do?
All that happened was that you were logged out of SYNCbits. When you are logged out your data visually disappears (think of it as a mail client – when you are logged in, you see your emails and when you log out you don’t see them, even though they are there). All you have to do is to simply log back into SYNCbits.
Don’t remember your password?
Please go to https://my.syncbits.com and reset your password using your security answers. Remember that the security answers are case sensitive.
If you have any issues please contact our tech support at [email protected]
Thank you for your time and understanding and please accept our deepest apologies about the caused inconvenience.
Update from 17 Aug 2018:
I am happy to announce that we’ve successfully replaced mcrypt with openssl, tested and migrated the fix to production. So, that’s one problem we won’t have again 🙂
Founder & CEO