Square-to-Square Online Sync Issues when using Square API

[cbstl]

@MayaP - Since you've been helpful several times before, I want to bring this to your attention, because I've been unable to get help via other avenues. Because of the way our business operates, we have a web app that uses the Square API to dynamically update our own pickup date modifiers on our items. I've posted about how it works in the past on these forums here:

https://community.squareup.com/t5/Food-and-Beverage/Bakery-Pre-orders/m-p/385318#M2131

It's been working perfectly for several years, up until a few months ago when the modifiers that get updated by the API calls would intermittently be incomplete in Square Online only. The contents of the modifier get updated correctly and show up right in the item in the Square Dashboard, but in the item on our Square Online store, the modifier would be missing elements. When that happens, the only way to get the modifier to subsequently show up correctly in the online store would be to make some other subsequent change to the associated item to force Square to sync everything to Square Online again. For a more detailed description of the problem and examples, you can see what I posted in the Square developer forums (since this involves use of the Square API):

https://developer.squareup.com/forums/t/possible-race-condition-between-square-to-square-online-sync...

On the Square Developer forums, the Square rep let me know that the team is aware of the problem, but he couldn't give me any more information or status than that and encouraged me to reach out to Square Customer Success. I had already done that though. On March 3rd, I had also opened a ticket via Square Email support. The Case# for that support request is #104713901. If you're able to look at that case, I'd appreciate it, because the "support" has been a disaster. More than a week later, this is the exact, nearly unintelligible initial response that I got:

This is Caren thanks a lot for contact us, in this I do apologizes for inconveniences that you're experiencing with the item and all related now we are working to solve the issues like known issues and glitches on the system the engineers will contact you about it and fee free to ask them about it, I hope you are fine and thanks for be part of Square  

I've been burned by Square email support before and the complete inability of that support person to communicate in coherent English gave me no confidence that we would get anywhere on what is a complex technical problem. After several more emails in which I provided our Square Online site link and provided screen shots of the problem, I stopped getting responses completely. The last email I got was on March 28th, despite me sending several follow up emails, the last few of which were just asking for the ticket to be closed so that I could leave feedback and it's still been complete radio silence. This is an absolutely abysmal "support" experience.

For the time being, I've had to put a very hacky change in our web app to work around this problem. Basically, the web app now has to regularly make an arbitrary change to an otherwise unused custom attribute on each item to force a sync of the item and all its modifiers from Square to Square Online to ensure that the pickup date modifiers show all their elements in our Square Online store. I do not want to leave that functionality in our web app forever though, because it's very much an ugly hack that is only necessary because Square-to-Square Online syncing isn't working right. I understand this problem is a bit of an outlier, but I just need someone at Square who is aware of the problem to be able to give me a status on when it's actually fixed and neither the developer forums, nor email support have been able to provide that. This has been very frustrating. Can you help?

[MayaP]

Hello @cbstl – Thanks for flagging this. Aside from this issue, I hope everything else is going well in the meantime.

I really appreciate your detailed explanation, and I completely understand how frustrating it is to have this issue lingering without a clear resolution. I’ve reached out to the API team for more insight and asked if we can help move things along. I’ll update you as soon as I hear back 🤞

I know it’s not easy to deal with this while running your business, and I truly appreciate your patience and understanding.

[MayaP]

I’ve shared the examples you provided in the forum (Nutella S’mores Bars and Double Decker) with our team. If you have any other examples, feel free to send them my way.

Thank you!

@cbstl 

[cbstl]

Thank you @MayaP . We've actually been seeing other issues as well, where Square randomly re-orders the modifiers on some of our items and then we have to go into the Dashboard and set them back to the correct order. I don't know if that issue is a related bug or not, but I wanted to take one thing at a time. 

[MayaP]

Oh no, sorry to hear there's another issue. I’m sure you’ve already checked this, but is everything on your end up to date? @cbstl 

[cbstl]

@MayaP - Can you be more specific about what you should be checked for being up-to-date? We use the Square Dashboard on the web (squareup.com) to maintain the item details and our Square Online store, so that is inherently up-to-date. We generally never use the Square for Retail app for updating anything about our items - we only use it for ringing up customers in-store.

[MayaP]

For the other issue, did you make any changes to "Double Decker"? Our team is seeing a different picture than the one in the forum? 

@cbstl 

[cbstl]

@MayaP - The screenshots that were posted were from weeks ago and those pickup date modifiers change at least every day, sometimes multiple times per day depending on what's ordered, so what's currently on the site is not going to match those screenshots. Furthermore, with my workaround in place in the web app, it is very unlikely to see the problem manifest now (that's the point of the workaround - we would not be able to operate without it because the pickup date modifiers randomly would be all screwed up). The problem is going to have to be reproduced by your technical staff in their own standalone/sandbox account. The steps to reproduce are:

1. Use the UpsertCatalogObjectRequest API call to change the elements of a modifier on an item (add or remove some elements). It doesn't seem to matter how many elements are in the modifier, but it might be more likely to occur and obvious if the modifier has a lot of elements (>30). 

2. Observe the elements of the modifier in the Square Dashboard and on the item in a Square Online store. The API call usually updates the elements in the Square Dashboard immediately, but the subsequent syncing to Square Online seems to take an arbitrarily longer amount of time (minutes or more). 

3. See if the elements in the modifier shown in the item in the Square Online store are incomplete compared to what's in the Square Dashboard. The problem is intermittent and seems to be a strange timing issue so you may need to repeat these steps several times until you see the problem occur.

Obviously, I don't know anything about the inner workings of Square, but my best guess is that there's a race condition between the API handler and the sync process, where the API handler might be indicating to Square Online that updates need to be synced for an object before it’s actually done committing the changes to the data source that Square Online pulls from. Then, depending on the timing of Square Online trying to pull the updated object data, it’s sometimes incomplete or missing because the API handler hasn’t finished committing the changes. It also could simply be a buggy Square-to-Square Online sync process that manifests for these modifiers because they get updated so often. 

[cbstl]

@MayaP - If it would be helpful, I could try to reproduce the problem on a test item in our account that we wouldn't actually sell. It would take some setup for me to try to do that though and I don't know how helpful that would be to your support folks. They could see an item that's in the misaligned state, but they wouldn't be able to observe the conditions that got it into that state like they could if they reproduce it themselves. Let me know if you want me to try to do that though and I'll give it a shot.

[MayaP]

Hey @cbstl, thank you so much for hanging in there. I just wanted to check in and let you know I’m not ignoring you — I’m still working with our team to investigate this. I really appreciate your patience!

[cbstl]

Hi @MayaP - Thanks for checking in on this, I do appreciate it. If there's anything I can do to help the team, please let me know.

[MayaP]

Thanks, a ticket has been filed for this @cbstl. So I can add it to the ticket, could you confirm which browser and browser version you’re using? Thank you.

[cbstl]

Hi @MayaP - We use Google Chrome exclusively and stay current with the latest version, which at the moment is Version 139.0.7258.128.

[MayaP]

Thank you! Here’s the case number attached to the ticket: 110267782.

@cbstl 

[cbstl]

Hi @MayaP - Thank you for initially opening that ticket. We received the following initial email for the ticket:

Thank you for sharing the details and examples regarding the issue with modifiers not syncing properly between your Square Dashboard and your Square Online store.

We’ve escalated this case to our engineering team for further review so they can investigate what might be causing these discrepancies. At this time, we don’t have an estimated timeline for a resolution, but please rest assured that the team is aware and looking into it.

In the meantime, if you notice this happening with any additional items, please feel free to share the specific item names and timestamps when the issue occurred. This information can help us provide more context to our engineering team during their investigation.

We’ll keep you updated as soon as we have more information to share.

We didn't respond, because the support person (Frank) said they would keep us updated as soon as they had some information. I responded to that email yesterday requesting that any updates on the ticket be sent to an additional email address (because I'm not the owner of the main account email and didn't want to miss updates on the ticket) and we subsequently got this message:

Please Create a New Case with Square Support

Thanks for your reply! It's been more than 2 weeks since we last heard from you, so we closed this case.

If you have additional questions, please create a new case with our Support Team by visiting the Support Center.

Additionally, feel free to visit Square's Seller Community to get ideas and help from other Square sellers.

That's very frustrating. We received no other follow-up nor any warning that the case would be automatically closed if we didn't respond within 2 weeks. I can't create a new support case myself, because that got me nowhere last time. Can you please get this ticket (#110267782) re-opened?

[MayaP]

Good Morning @cbstl I didn't realize that was their process either. That is frustrating! 

I have sent a message to that advocate and requested that the case be opened. I will keep you updated! Thanks for always being so patient. 

[cbstl]

Thank you @MayaP - When it is re-opened, can you find out if we will need to send an otherwise unnecessary email every two weeks to keep the ticket open or if it can be set somehow to not automatically close?

[MayaP]

I sent another follow-up to the advocate, but it looks like they won’t be in until tomorrow. I’m off on Fridays, but I’ll pop in to check for a reply and will keep you updated.

@cbstl 

[MayaP]

Hi @cbstl apologies for the delay in getting back to you. The advocate confirmed the case has been reopened and I can see they emailed you. They didn’t mention you need to write in, but as noted in their email, the system did auto-close the case. I’m not sure why that happens, but our team will continue to investigate since the ticket is open and ongoing. Thank you for your understanding and for continuing to work with us on this.

[cbstl]

Thank you @MayaP . I did see the email from the advocate (Frank) and will respond to him with further questions. I appreciate you helping with that!