S3 LS: Error during pagination: The same next token was received twice #3917
Comments
AWS Version:
aws-cli/1.16.101 Python/3.7.2 Linux/4.20.5-200.fc29.x86_64 botocore/1.12.91
Command:
aws s3 ls s3://test-test/ —profile wasabi —endpoint-url=https://s3.wasabisys.com
Error:
Error during pagination: The same next token was received twice:
Bucket has > 100.000 objects.
This seems to have come up suddenly, also tried running version 1.16.62 and 1.16.80 but i get the same error.
I tried to look for duplicate filenames in the bucket but found none.
The text was updated successfully, but these errors were encountered:
@HyperDevil — Thanks for reporting this issue. We are investigating it. It may also be related to . [removed related GitHub issues after confirm they were not relate to this issue]
Could be related although this is a regular «s3 ls» command and not (i assume) used for filter-log-events?
Wasabi (my S3 provider) is now also looking at the server logs in the meantime.
@HyperDevil — Thanks for the feedback. Correct the commands are different but the issue related to pagination is similar. I’m still investigating this issue but wanted to collect additional information to analyze the problem more.
Is this an intermittent issue or does this happen all of the time?
Are you able to run it multiple times with the same results?
@HyperDevil — I have tried to reproduce this issue with different versions of the CLI using AWS endpoints. The CLI appears to be working as expected and the problem might be isolated to the —endpoint-url=https://s3.wasabisys.com . It appears you might be using their documentation as guidance.
When you said «This seems to have come up suddenly» does the mean it worked as expected under a lower version of the CLI?
Are the 100,000 items stored at the root of the bucket?
Any chance you can upload the rest of the debug output?
Yes i am using a different S3 storage target, although in this case i don’t think it matters.
I have made case with Wasabi to check their servers logs, but i don’t expect anything on the server side to be broken. ( i will update this case when i get feedback from them)
It happened suddenly without any change in version number, i suspect it has to do with the amount of files. The amount of files in my bucket always increases. The files are named: urn:oid:increasingnumber
At this moment this problem is a permanent one, it completely stopped working.
Also tried version 1.16.62 and 1.16.80, but it fails with the same error.
I checked the bucket and did not find duplicate objects.
The debug output is pasted above, that is all the output i get with the debug flag.
@HyperDevil — Thank you for your feedback. It is possible information retrieved from using the aws history show command will provide more details to help isolate the problem. It will have to be enabled first, the problem reproduce, and then run the command.
The output has of course been cut down since it’s too long.
There are no appearant errors, it just doesnt continue with the next API call.
Error Bad Request: Missing parameter: ‘code’
When setting up Site Kit, I click on SIGN IN WITH GOOGLE I choose the account to connect with and then when I click Continue on the screen saying Site Kit wants additional access to your Google Account I get the same
Error: Bad Request: Missing parameter: ‘code’ message that other users have reported and I don’t get connected to the account.
I have tried the Health Check & Troubleshooting, and I’ve tried using a default theme and the error persists.
- This topic was modified 8 months, 1 week ago by JohnJay .
- This topic was modified 8 months, 1 week ago by JohnJay .
- This topic was modified 8 months, 1 week ago by JohnJay .
- This topic was modified 8 months, 1 week ago by JohnJay .
Thanks for reaching out, and also sharing details of the troubleshooting efforts you’ve undertaken. As we’re currently investigating reports of this it would be great if you could share the following:
- Your Site Health information. Within the form please include your current hosting provider.
- When checking the Site Kit dashboards from a Chrome browser incognito window do you see any browser console errors on the same screen where this notice appears? If so please share any such errors.
- Would it be possible for you to share a screen recording of your experience setting up the plugin, that shows your full experience? You can use a service such as Loom or RecordIt to share. Feel free to include this in the above Site Health information form.
Let me know if you have any questions with the above. Once we have this information we can investigate this further or provide you with some troubleshooting suggestions, which may include attempting setup from another site on the same hosting platform or using another Google account to attempt setup.
@jjacoberger Out of curiosity are you using a specific browser to attempt setup? If so please check does the same occur from a Chrome browser.
I completed your form as requested. I received the same error using Chrome, Edge and Brave.
I was finally able to complete the setup using Firefox in Incognito mode.
Many thanks for sharing, and glad to hear you’ve been able to successfully connect using Firefox. I’ve checked your details and as nothing stands out I suspect this may be browser specific. By any change do you have other WordPress sites where you can check for the same setup experience with, using the Brave and then Chrome browser?
Yes, I have several other sites – I can try them later on this afternoon and let you know what happens.
That would be great, thank you! We’re curious to know if the same always occurs with the Brave browser while then on the same sites does the same occur when switching to Chrome in particular.
@jjacoberger Just following up with you to check whether you’ve had an opportunity to check your other sites for the same error?
I actually finally did last evening – sorry for the delay but serious life events interfered.
On the first site:
- I was using the Brave Browser
- I entered the Google account and password to connect
- I agreed to continue granting access to the account
- I received the error
- I logged into the site Admin Dashboard again using Chrome
- I followed all prompts to connect to the Google account
- I received the error again
- I logged into the site using Firefox
- I followed all the prompts enter login credential and connect to the Google account
- I connected Site Kit successfully with Firefox
On the second site:
On the third site:
Using the Brave browser, I was able to Connect Site Kit with a Google account successfully the first time I tried.
Interestingly, the first and third sites, where one was successful on the first try and the other was not, are exact duplicates of each other hosted by the same company.
Many thanks for the update, and no problem about the delay. I hope all is well regarding your reasoning.
In relation to the first and third sites there is a particular configuration within the Brave browser which we’ve been able to recreate the same error message with when enabled (Shields > Look and feel > Auto-redirect AMP pages). I wonder did you have this enabled at any stage. I can’t be sure why the results were inconsistent, however, it’s possible this configuration may not always trigger or detect AMP URLs.
On the second site do you recall whether you were using the Brave browser when you encountered this error? If so please share whether you were able to successfully connect when switching to Chrome.
Yes, on the second site, I received the error using brave. After I got the error with Brave, I switched to Chrome and got the same error. I connected on the third attempt with Firefox.
Many thanks for the update. I’ll add your notes to the related GitHub issue, where we’ve been adding details on this error, which other users also reported.
While I’ll close this support topic now given you’re successfully connected please inform me if you have any AMP related Chrome browser extensions active, such as the Redirect AMP to HTML extension.
I ask as the error you’re encounter is something we’ve been able to recreate using such extensions, or browser related configurations regarding AMP.
App Store Connect Operation Error
An error occurred saving your changes to the Apple database. This problem may be a transient issue on the Apple side. If the problem persists for more than an hour, please contact your iTunes representative. (1129)
Xcode version 12.1
Replies
We had the same error some hours ago, now it changed to
Unable to process validateAssets request at this time due to a general error (1017)
We have same error forcouple of hours.. 🙁
An error occurred saving your changes to the Apple database. This problem may be a transient issue on the Apple side. If the problem persists for more than an hour, please contact your iTunes representative. (1129)
Please solve asap.
An error occurred saving your changes to the Apple database. This problem may be a transient issue on the Apple side. If the problem persists for more than an hour, please contact your iTunes representative. (1129)
Sorry if I’m coming in here brimming with negativity, but it’s never fun to wake up in the morning and see issues like this.
The notarization requirement is bad enough as is, given that, in my opinion, it adds essentially no additional security to the platform, especially since there’s already been at least one case of malware fooling Apple’s notary into notarizing it. (Why not just check a quarantined executable for malware client side, rather than use a notary?!)
But the fact that the notary service can’t maintain consistent uptime just makes it painful. I can’t send a build to a client now.
Strangely Apples system status page still lists «Developer ID Notary Service» as green.
Error Bad Request: Missing parameter: ‘code’
When setting up Site Kit, I click on SIGN IN WITH GOOGLE I choose the account to connect with and then when I click Continue on the screen saying Site Kit wants additional access to your Google Account I get the same
Error: Bad Request: Missing parameter: ‘code’ message that other users have reported and I don’t get connected to the account.
I have tried the Health Check & Troubleshooting, and I’ve tried using a default theme and the error persists.
- This topic was modified 8 months, 1 week ago by JohnJay .
- This topic was modified 8 months, 1 week ago by JohnJay .
- This topic was modified 8 months, 1 week ago by JohnJay .
- This topic was modified 8 months, 1 week ago by JohnJay .
Thanks for reaching out, and also sharing details of the troubleshooting efforts you’ve undertaken. As we’re currently investigating reports of this it would be great if you could share the following:
- Your Site Health information. Within the form please include your current hosting provider.
- When checking the Site Kit dashboards from a Chrome browser incognito window do you see any browser console errors on the same screen where this notice appears? If so please share any such errors.
- Would it be possible for you to share a screen recording of your experience setting up the plugin, that shows your full experience? You can use a service such as Loom or RecordIt to share. Feel free to include this in the above Site Health information form.
Let me know if you have any questions with the above. Once we have this information we can investigate this further or provide you with some troubleshooting suggestions, which may include attempting setup from another site on the same hosting platform or using another Google account to attempt setup.
@jjacoberger Out of curiosity are you using a specific browser to attempt setup? If so please check does the same occur from a Chrome browser.
I completed your form as requested. I received the same error using Chrome, Edge and Brave.
I was finally able to complete the setup using Firefox in Incognito mode.
Many thanks for sharing, and glad to hear you’ve been able to successfully connect using Firefox. I’ve checked your details and as nothing stands out I suspect this may be browser specific. By any change do you have other WordPress sites where you can check for the same setup experience with, using the Brave and then Chrome browser?
Yes, I have several other sites – I can try them later on this afternoon and let you know what happens.
That would be great, thank you! We’re curious to know if the same always occurs with the Brave browser while then on the same sites does the same occur when switching to Chrome in particular.
@jjacoberger Just following up with you to check whether you’ve had an opportunity to check your other sites for the same error?
I actually finally did last evening – sorry for the delay but serious life events interfered.
On the first site:
- I was using the Brave Browser
- I entered the Google account and password to connect
- I agreed to continue granting access to the account
- I received the error
- I logged into the site Admin Dashboard again using Chrome
- I followed all prompts to connect to the Google account
- I received the error again
- I logged into the site using Firefox
- I followed all the prompts enter login credential and connect to the Google account
- I connected Site Kit successfully with Firefox
On the second site:
On the third site:
Using the Brave browser, I was able to Connect Site Kit with a Google account successfully the first time I tried.
Interestingly, the first and third sites, where one was successful on the first try and the other was not, are exact duplicates of each other hosted by the same company.
Many thanks for the update, and no problem about the delay. I hope all is well regarding your reasoning.
In relation to the first and third sites there is a particular configuration within the Brave browser which we’ve been able to recreate the same error message with when enabled (Shields > Look and feel > Auto-redirect AMP pages). I wonder did you have this enabled at any stage. I can’t be sure why the results were inconsistent, however, it’s possible this configuration may not always trigger or detect AMP URLs.
On the second site do you recall whether you were using the Brave browser when you encountered this error? If so please share whether you were able to successfully connect when switching to Chrome.
Yes, on the second site, I received the error using brave. After I got the error with Brave, I switched to Chrome and got the same error. I connected on the third attempt with Firefox.
Many thanks for the update. I’ll add your notes to the related GitHub issue, where we’ve been adding details on this error, which other users also reported.
While I’ll close this support topic now given you’re successfully connected please inform me if you have any AMP related Chrome browser extensions active, such as the Redirect AMP to HTML extension.
I ask as the error you’re encounter is something we’ve been able to recreate using such extensions, or browser related configurations regarding AMP.