Skip to content

chore(coderd/notifications): expand golden file testing for notifications #15032

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 49 commits into from
Oct 14, 2024

Conversation

SasSwart
Copy link
Contributor

@SasSwart SasSwart commented Oct 10, 2024

This PR aims to close #14913.

It expands the golden files for the notifier to include the entire payload serialised as JSON.

@SasSwart SasSwart requested a review from mafredri October 11, 2024 10:57
@SasSwart SasSwart marked this pull request as ready for review October 11, 2024 13:09
Copy link
Member

@mafredri mafredri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work on increasing the test coverage. I didn't look too closely at the actual rendered output as I lack a bit of context, but perhaps @dannykopping or @mtojek has more thoughts on that.

@SasSwart SasSwart requested a review from mtojek October 11, 2024 13:45
Copy link
Member

@mtojek mtojek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this is in a good shape 👍 Left a couple of nit-picks to take a look & reply. Feel free to ignore them or postpone for later!

Nice job, Sas 👍 👍

}

func normalizeGoldenWebhook(content []byte) []byte {
const constantUUID = "00000000-0000-0000-0000-000000000000"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

{
  "_version": "1.1",
  "msg_id": "00000000-0000-0000-0000-000000000000",
  "payload": {
    "_version": "1.1",
    "notification_name": "Workspace Autobuild Failed",
    "notification_template_id": "00000000-0000-0000-0000-000000000000",
    "user_id": "00000000-0000-0000-0000-000000000000",

Ok, I know why you did this but I'm wondering if there is a way to keep stable uuids written to the golden file 🤔. It could help find out why some labels change, and prevent bugs in the future.

PS. If this is difficult->impossible, it can be postponed for later (add FIXME label).

Copy link
Contributor Author

@SasSwart SasSwart Oct 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

keep stable uuids written to the golden file

What does stable mean in this context?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean, we could replace dynamic UUIDs with some const/known values rather than zeros. For instance:

user_id -> ceb6824e-7fe1-4000-aad9-b80bb141560a

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

from your comment below I infer that stable means constant and applies to the notification_template_id

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK. Thanks

Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset=UTF-8

Hi Bobby,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't know that we could render two content-types. Curious, if we can store golden results in separate files?

Copy link
Contributor Author

@SasSwart SasSwart Oct 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious, if we can store golden results in separate files?

The premise that I adopted for this PR was one of integration testing. This output is meant to represent exactly what would have been sent to the customer. I did not make them render in the same file. This is how it has been. This is what we currently send to customers.

To separate them would, in my opinion, introduce artificial opportunity for drift from actual behaviour. These tests are most reliable if they not only mimick, but actually dogfood the actual system's behaviour.

If you'd like me to change it to be multiple files, I would be happy to. Would you mind helping me understand the benefit though?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok! If there are no benefits, then let's leave them as is. Maybe replace .html.golden with .smtp.golden as this is not just HTML file, but also Plaintext and headers.

Would you mind helping me understand the benefit though?
My original intention was keep HTML in .html.golden files, and Plaintext in .txt.golden

"payload": {
"_version": "1.1",
"notification_name": "User account activated",
"notification_template_id": "00000000-0000-0000-0000-000000000000",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more comment:

Can we keep notification_template_id as is, and not zero it? These values are const/stable.

Copy link
Member

@mtojek mtojek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's get this large PR merged, nit-picks can be done in a follow-up.

@SasSwart SasSwart enabled auto-merge (squash) October 14, 2024 12:30
@SasSwart SasSwart merged commit 208ed1e into main Oct 14, 2024
26 checks passed
@SasSwart SasSwart deleted the jjs/14913 branch October 14, 2024 12:34
@github-actions github-actions bot locked and limited conversation to collaborators Oct 14, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Notifications: verify rendered action
3 participants