Extract JSON value from Web Scenario Step

[highpeak]

I have a web scenario where I need to use a token from one request in the next step. In my first step I have my request and the following:

Variables
Name: {token}
Value: regex: "token": "(.*)",

Example JSON:


{
"id": "1234",
"token:: "abcd",
"access: "true"
}

I get an error saying "cannot extract the value token from response"

I'm pretty sure I'm getting my regex wrong, with missing escaped quotes or something, but I've tried as many combinations as I can think of and I've validated that this is a valid PCRE regex expression.

[tim.mooney]

The example JSON you show has two ':' symbols (and no ") after token, rather than one. Is that just a typo in your example, or is it actually what your application is generating?

Keep in mind too that regexes by default are greedy: they will match as much as possible. If your regex is too general, it can cause it to match far more than you want it to. In your case, the .* could match a very long line with lots of other fields in it, for example:

"token": "what you want to match", "other-stuff": "matches this too", "more-stuff": "still matches this", "even-more": "this can go on till the last double-quote on the line",

You may want to try a regular expression that's more specific, to ensure that it stops matching at the first " it sees. There are various ways to do that. I would try something like

regex: "token": "([^"]+)"

That's basically "one or more of anything except double-quote". If that doesn't work, then if you know that the token has a limited set of possible characters (say A-Z, a-z, 0-9, the "-" and the "_" character), then you could use a character class that matches those.

Also, one separate thing to consider. I don't think this is your issue, but when I've had to extract a session variable from a multi-step web scenario, I've been extracting it from the web page the server sends (i.e. the text/html page). You're trying to extract it not from a web page but from a JSON response type. I don't know if there are any subtle differences when doing that. Hopefully someone that has deeper experience with web scenarios can follow up if there is anything special/different about using a web scenario where the returned data is e.g. application/json rather than text/html. Again, I don't think this is your problem, but I also don't know if this makes a difference or not.

The web scenario documentation could really use more examples.

[highpeak]

Firstly Tim, thanks for catching my bad typos. I've added the real JSON example below:

{
"token": "eyJhbGciOiJodHRwOi8vd3d3LnczLm9yZy8yMDAxLzA0. Etx5 aVjVLlxP8Y",
"cameras": [
88175,
88517,
88576
],
"isSuccessful": true,
"errors": [],
"message": "OK",
"statusCode": 200
}

I user your suggestion of

regex: "token": "([^"]+)"

But I still get the error

{token}=regex: "token": "([^"]+)"": cannot extract the value of "{token}" from response

Do I need to wrap the regex in additional quotes in the Web Scenario variable field? If not, then I can only assume I'm butting up against your "parse a JSON response issue".

If I increase the log level for the http poller, however, I see this:

2323:20200303:090517.415 process_httptest() page.data from https://xxx.co.za/api/authenticate:'HTTP/1.1 200 OK                                                            
Transfer-Encoding: chunked                                                                                                                                                                                                 
Content-Type: application/json; charset=utf-8                                                                                                                                                                       
Server: Kestrel                                                                                                                                                                                                            
Request-Context: appId=cid-v1:7ec5f458-4dbe-4dfc-a646-d27f4c48146f                                                                                                                 
X-Powered-By: ASP.NET                                                                                                                                                                                          
Set-Cookie: ARRAffinity=fd251da5b9564b197bdb67d3c6c32306c5647396fd4523286b8a1cac4dc7efe8;Path=/;HttpOnly;Domain=xxxx.co.za
Date: Tue, 03 Mar 2020 07:05:16 GMT                                                                                                                                                                                        x
                                                                                                                                                                                                                          x
["token":"eyJhbGciOiJodHRwOi8vd3[...]

So I know I'm getting data back that I can parse.

Also, if I put a very simple regex: (.*) in then my variable {token} is assigned "200 OK"

I'm really stumped.

[tim.mooney]

My comment about the application/json type was just speculation; I don't know if that's part of the problem (now seeing your additional data, I suspect *not*), but it was something I was thinking about when looking at your problem.

What happens if you try this regex, instead:

regex:"token":"([^"]+)"

All that does is remove the spaces, *in particular* the space between "token": and the start of the value.

If that doesn't work, then assuming the contents of any token that's generated will look like the one you show above (lower case, upper case, numbers, the . character, and space), you could also try:

regex:"token":"([A-Za-z0-9\. ]*)"

Please try one or both of these and let the forums know if that makes any difference.