<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; font-size: 14px; font-family: Calibri, sans-serif;">
<div>Justin, just to clarify, my local tests show that the JTI is <font color="#ff0000">
<b>NOT</b></font> in the ID token and they <font color="#ff0000"><b>ARE</b></font> in the access tokens. See below decoded tokens that I just got (1.1.16):</div>
<div style="color: rgb(0, 0, 0);"><br>
</div>
<div style="color: rgb(0, 0, 0);">
<div><span style="font-size: medium;">2015/Sep/01 11:15:15</span>
<h1 style="font-family: '';">Access Token</h1>
<span style="font-size: medium;">{ </span><br style="font-family: ''; font-size: medium;">
<span style="font-size: medium;"> "aud": [ </span><br style="font-family: ''; font-size: medium;">
<span style="font-size: medium;"> "growth_chart" </span><br style="font-family: ''; font-size: medium;">
<span style="font-size: medium;"> ], </span><br style="font-family: ''; font-size: medium;">
<span style="font-size: medium;"> "iss": "http://localhost:8080/ldap-openid-connect-server/", </span><br style="font-family: ''; font-size: medium;">
<span style="font-size: medium;"> "exp": 1441120575, </span><br style="font-family: ''; font-size: medium;">
<span style="font-size: medium;"> "iat": 1441120515, </span><br style="font-family: ''; font-size: medium;">
<span style="font-size: medium;"> "jti": "54384e60-1ef6-4612-9ebc-374023ee7687" </span><br style="font-family: ''; font-size: medium;">
<span style="font-size: medium;">}</span>
<h1 style="font-family: '';">ID Token</h1>
<span style="font-size: medium;">{ </span><br style="font-family: ''; font-size: medium;">
<span style="font-size: medium;"> "iss": "http://localhost:8080/ldap-openid-connect-server/", </span><br style="font-family: ''; font-size: medium;">
<span style="font-size: medium;"> "sub": "CATS_dashsrv", </span><br style="font-family: ''; font-size: medium;">
<span style="font-size: medium;"> "aud": [ </span><br style="font-family: ''; font-size: medium;">
<span style="font-size: medium;"> "growth_chart" </span><br style="font-family: ''; font-size: medium;">
<span style="font-size: medium;"> ], </span><br style="font-family: ''; font-size: medium;">
<span style="font-size: medium;"> "exp": 1441120575, </span><br style="font-family: ''; font-size: medium;">
<span style="font-size: medium;"> "iat": 1441120515, </span><br style="font-family: ''; font-size: medium;">
<span style="font-size: medium;"> "kid": "rsa1" </span><br style="font-family: ''; font-size: medium;">
<span style="font-size: medium;">}</span></div>
</div>
<div style="color: rgb(0, 0, 0);"><span style="font-size: medium;"><br>
</span></div>
<div style="color: rgb(0, 0, 0);">Regards,</div>
<div style="color: rgb(0, 0, 0);">Luiz</div>
<div style="color: rgb(0, 0, 0);"><br>
</div>
<span id="OLK_SRC_BODY_SECTION" style="color: rgb(0, 0, 0);">
<div style="font-family:Calibri; font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style="font-weight:bold">From: </span>Justin Richer <<a href="mailto:jricher@mit.edu">jricher@mit.edu</a>><br>
<span style="font-weight:bold">Date: </span>Tuesday, September 1, 2015 at 11:34 AM<br>
<span style="font-weight:bold">To: </span>Luiz Omori <<a href="mailto:luiz.omori@dm.duke.edu">luiz.omori@dm.duke.edu</a>><br>
<span style="font-weight:bold">Cc: </span>"<a href="mailto:mitreid-connect@mit.edu">mitreid-connect@mit.edu</a>" <<a href="mailto:mitreid-connect@mit.edu">mitreid-connect@mit.edu</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [mitreid-connect] Exception during cleanup<br>
</div>
<div><br>
</div>
<div>
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
The tokens are supposed to be unique anyway, so that’s not good to see. But seeing the problem in this context, when it’s not the exact same request creating erroneous multiple tokens, has shed new light on things.
<div class=""><br class="">
</div>
<div class="">Adding a JTI will probably fix this. ID tokens already have them:
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class=""><a href="https://github.com/mitreid-connect/OpenID-Connect-Java-Spring-Server/blob/9ba1a78d09eec4fd6b142276118b39170c450cb4/openid-connect-server/src/main/java/org/mitre/openid/connect/token/ConnectTokenEnhancer.java#L101" class="">https://github.com/mitreid-connect/OpenID-Connect-Java-Spring-Server/blob/9ba1a78d09eec4fd6b142276118b39170c450cb4/openid-connect-server/src/main/java/org/mitre/openid/connect/token/ConnectTokenEnhancer.java#L101</a></div>
<div class=""><br class="">
</div>
<div class="">And refresh tokens have them:</div>
<div class=""><br class="">
</div>
<div class=""><a href="https://github.com/mitreid-connect/OpenID-Connect-Java-Spring-Server/blob/22c05ec51bce775e7f6c737089d85a4258be1f6f/openid-connect-server/src/main/java/org/mitre/oauth2/service/impl/DefaultOAuth2ProviderTokenService.java#L220" class="">https://github.com/mitreid-connect/OpenID-Connect-Java-Spring-Server/blob/22c05ec51bce775e7f6c737089d85a4258be1f6f/openid-connect-server/src/main/java/org/mitre/oauth2/service/impl/DefaultOAuth2ProviderTokenService.java#L220</a></div>
<div class=""><br class="">
</div>
<div class="">So I could have *sworn* that access tokens had them too! But looking at the generation function, they apparently don’t:</div>
<div class=""><br class="">
</div>
<div class=""><a href="https://github.com/mitreid-connect/OpenID-Connect-Java-Spring-Server/blob/22c05ec51bce775e7f6c737089d85a4258be1f6f/openid-connect-server/src/main/java/org/mitre/oauth2/service/impl/DefaultOAuth2ProviderTokenService.java#L143" class="">https://github.com/mitreid-connect/OpenID-Connect-Java-Spring-Server/blob/22c05ec51bce775e7f6c737089d85a4258be1f6f/openid-connect-server/src/main/java/org/mitre/oauth2/service/impl/DefaultOAuth2ProviderTokenService.java#L143</a></div>
<div class=""><br class="">
</div>
<div class="">We’ll also need to check special tokens like registration tokens and UMA tokens that are made from their own services.</div>
<div class=""><br class="">
</div>
<div class="">I think that’s a worthwhile addition and it’s one that’s easy to put in and it should be easy to test. It should probably also be backported to 1.1. I’ve added an issue for it here:</div>
<div class=""><br class="">
</div>
<div class=""><a href="https://github.com/mitreid-connect/OpenID-Connect-Java-Spring-Server/issues/900" class="">https://github.com/mitreid-connect/OpenID-Connect-Java-Spring-Server/issues/900</a></div>
<div class=""><br class="">
</div>
<div class="">If someone would like to contribute SQL changes to add indices and constraints to the token tables where appropriate as well, please do! The database has not been really updated like that. As a rule we don’t update the database schema within a
release cycle (so 1.0, 1.1, and 1.2 all have “stable” schemas), but we’ve made an exception in the past just for indexes and constraints exactly like this, since they can be easily added on top of production systems without affecting valid data if people desire.</div>
<div class=""><br class="">
</div>
<div class="">Thanks everyone, this is starting to make a little bit more sense.</div>
<div class=""> — Justin</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
<div class="">
<div>
<blockquote type="cite" class="">
<div class="">On Sep 1, 2015, at 10:50 AM, Luiz Omori <<a href="mailto:luiz.omori@duke.edu" class="">luiz.omori@duke.edu</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; font-size: 14px; font-family: Calibri, sans-serif;" class="">
<div class="">OK, I managed to reproduce the problem: it’s related to duplicated ID tokens being created when multiple
<b class="">similar</b> requests are received within the same second. In our case we use the Resource Owner flow for some legacy web services calls and that exacerbates the problem. At some point the code does a select on the access_token table using the token_value
column and checks for number of rows returned, enforcing 1.</div>
<div class=""><br class="">
</div>
<div class="">This token appears 38 times in the access_token table when we ran a stress test that makes 50 parallel Resource Owner requests (test environment):</div>
<div class=""><br class="">
</div>
<div class="">eyJraWQiOiJyc2ExIiwiYWxnIjoiUlMyNTYifQ.eyJpc3MiOiJodHRwOlwvXC9sb2NhbGhvc3Q6ODA4MFwvbGRhcC1vcGVuaWQtY29ubmVjdC1zZXJ2ZXJcLyIsInN1YiI6IkNBVFNfZGFzaHNydiIsImF1ZCI6WyJncm93dGhfY2hhcnQiXSwiZXhwIjoxNDQxMTE0NTMzLCJpYXQiOjE0NDExMTQ0NzMsImtpZCI6InJzYTEifQ.NVSFBd9onhEUgMDC46KAJx5M_CaBy-INuy0MqodLqOCO7YQ5N9ZARG9BKemjv8G0Aj8aIYOa-CJeUYycv6Uk1qqVTcXX9jgpX7-f9xK4T4MxcOhhTwRqkvT8l7yWL5uAm1uG7lcKKHw6h2RGed4x9zej1Zb0PFiHTZHkPRl6OWwJvmaZySd0CZM3VVVtW4HIB45vNURE--ee5zkH1ezoJOfR0JlFFEaQfcTnW-PGBcNri6huk1fosUwLxoZE8-YGjwWVzKyy_1qmZBhS9Yg1ch6uXuxaXjG96_ITZQVoIyFwWyMi3rXOiwdgAzN_7aZOt8HSxNm7OUqFxZEr6JChbw</div>
<div class=""><br class="">
</div>
<div class="">The concern is the growth of that table and since at least in MySQL version the token_value is not indexed (we could fix it locally) potential slow down over time.</div>
<div class=""><br class="">
</div>
<div class="">Is there any workaround that we could use on our production system? Unfortunately, as opposed to my local system, our production environment is using HSSQLDB which most likely will prevent us from running delete SQL commands while the system is
running. </div>
<div class=""><br class="">
</div>
<div class="">Maybe we can add JTI to the ID tokens to make them unique?</div>
<div class=""><br class="">
</div>
<div class="">Regards,</div>
<div class="">Luiz</div>
<div class=""><br class="">
</div>
<span id="OLK_SRC_BODY_SECTION" class="">
<div style="font-family: Calibri; font-size: 11pt; text-align: left; border-width: 1pt medium medium; border-style: solid none none; padding: 3pt 0in 0in; border-top-color: rgb(181, 196, 223);" class="">
<span style="font-weight:bold" class="">From: </span>Luiz Omori <<a href="mailto:luiz.omori@dm.duke.edu" class="">luiz.omori@dm.duke.edu</a>><br class="">
<span style="font-weight:bold" class="">Date: </span>Monday, August 31, 2015 at 4:50 PM<br class="">
<span style="font-weight:bold" class="">To: </span>"<a href="mailto:mitreid-connect@mit.edu" class="">mitreid-connect@mit.edu</a>" <<a href="mailto:mitreid-connect@mit.edu" class="">mitreid-connect@mit.edu</a>><br class="">
<span style="font-weight:bold" class="">Subject: </span>Exception during cleanup<br class="">
</div>
<div class=""><br class="">
</div>
<div class="">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; font-size: 14px; font-family: Calibri, sans-serif;" class="">
<div class="">Hi,</div>
<div class=""><br class="">
</div>
<div class="">We have a system in production that is logging some exceptions. This is happening in DefaultOAuth2ProviderTokenService.clearExpiredTokens. I’ve been trying to track that down but so far no luck reproducing it on my development environment. Has
anybody seem this? Is it benign or something we should be concerned about? </div>
<div class=""><br class="">
</div>
<div class="">
<div class=""><span class="Apple-tab-span" style="white-space:pre"></span>@Override</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"></span>public void clearExpiredTokens() {</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"></span>logger.info("Cleaning out all expired tokens");</div>
<div class=""><br class="">
</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"></span>Collection<OAuth2AccessTokenEntity> accessTokens = getExpiredAccessTokens();</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"></span>logger.info("Found " + accessTokens.size() + " expired access tokens");</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"></span>for (OAuth2AccessTokenEntity oAuth2AccessTokenEntity : accessTokens) {</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"></span>try {</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"></span>revokeAccessToken(oAuth2AccessTokenEntity);</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"></span>} catch (IllegalArgumentException e) {</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"></span>//An ID token is deleted with its corresponding access token, but then the ID token is on the list of expired tokens as well and there is</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"></span>//nothing in place to distinguish it from any other.</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"></span>//An attempt to delete an already deleted token returns an error, stopping the cleanup dead. We need it to keep going.</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"></span>}</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"></span>}</div>
<div class="">…</div>
</div>
<div class=""><br class="">
</div>
<div class="">
<div class="">2015Aug27 16:38:49,564: ERROR: org.springframework.scheduling.support.TaskUtils$LoggingErrorHandler - Unexpected error occurred in scheduled task.</div>
<div class="">java.lang.IllegalStateException: Expected single result, got 2</div>
<div class=""> at org.mitre.util.jpa.JpaUtil.getSingleResult(JpaUtil.java:36)</div>
<div class=""> at org.mitre.oauth2.repository.impl.JpaOAuth2TokenRepository.getAccessTokenByValue(JpaOAuth2TokenRepository.java:62)</div>
<div class=""> at org.mitre.oauth2.repository.impl.JpaOAuth2TokenRepository.removeAccessToken(JpaOAuth2TokenRepository.java:79)</div>
<div class=""> at org.mitre.oauth2.repository.impl.JpaOAuth2TokenRepository$$FastClassBySpringCGLIB$$145026dd.invoke(<generated>)</div>
<div class=""> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)</div>
<div class=""> at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:700)</div>
<div class=""> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)</div>
<div class=""> at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:96)</div>
<div class=""> at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:260)</div>
<div class=""> at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:94)</div>
<div class=""> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)</div>
<div class=""> at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:633)</div>
<div class=""> at org.mitre.oauth2.repository.impl.JpaOAuth2TokenRepository$$EnhancerBySpringCGLIB$$e7092f0b.removeAccessToken(<generated>)</div>
<div class=""> at org.mitre.oauth2.service.impl.DefaultOAuth2ProviderTokenService.revokeAccessToken(DefaultOAuth2ProviderTokenService.java:380)</div>
<div class=""> at org.mitre.oauth2.service.impl.DefaultOAuth2ProviderTokenService.clearExpiredTokens(DefaultOAuth2ProviderTokenService.java:411)</div>
<div class=""> at sun.reflect.GeneratedMethodAccessor210.invoke(Unknown Source)</div>
<div class=""> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)</div>
<div class=""> at java.lang.reflect.Method.invoke(Unknown Source)</div>
<div class=""> at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:64)</div>
<div class=""> at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:53)</div>
<div class=""> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)</div>
<div class=""> at java.util.concurrent.FutureTask.runAndReset(Unknown Source)</div>
<div class=""> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)</div>
<div class=""> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)</div>
<div class=""> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)</div>
<div class=""> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)</div>
<div class=""> at java.lang.Thread.run(Unknown Source)</div>
</div>
<div class=""><br class="">
</div>
<div class="">Regards,</div>
<div class="">Luiz</div>
</div>
</div>
</span></div>
_______________________________________________<br class="">
mitreid-connect mailing list<br class="">
<a href="mailto:mitreid-connect@mit.edu" class="">mitreid-connect@mit.edu</a><br class="">
<a href="http://mailman.mit.edu/mailman/listinfo/mitreid-connect">http://mailman.mit.edu/mailman/listinfo/mitreid-connect</a><br class="">
</div>
</blockquote>
</div>
<br class="">
</div>
</div>
</div>
</div>
</div>
</span>
</body>
</html>