Serializable objects should be deserializable between minor versions

[jkubrynski]

Currently after #1945 serialVersionUid is set for whole project, which means even if particular class is not changed between releases it'll be impossible to deserialize it due to serialVersionUid update. That could be solved by keeping serialVersionUid separate for each class and update it only after the class is changed. Also providing some tests to check compatibility.

[gonzalad]

This issue affects users when using spring session with java serialization (i.e. Spring Redis with Java serialization)

i.e. when upgrading our application to a new version of Spring Security on a Spring Session powered app using redis, we had the following error:

org.springframework.data.redis.serializer.SerializationException: Cannot deserialize; nested exception is org.springframework.core.serializer.support.SerializationFailedException: Failed to deserialize payload. Is the byte array a result of corresponding serialization for DefaultDeserializer?; nested exception is java.io.InvalidClassException: org.springframework.security.web.savedrequest.DefaultSavedRequest; local class incompatible: stream classdesc serialVersionUID = 8713433498670862907, local class serialVersionUID = 6412739018021506926
	org.springframework.data.redis.serializer.JdkSerializationRedisSerializer.deserialize(JdkSerializationRedisSerializer.java:82)
	org.springframework.data.redis.core.AbstractOperations.deserializeHashValue(AbstractOperations.java:338)
	org.springframework.data.redis.core.AbstractOperations.deserializeHashMap(AbstractOperations.java:282)
	org.springframework.data.redis.core.DefaultHashOperations.entries(DefaultHashOperations.java:227)
	org.springframework.data.redis.core.DefaultBoundHashOperations.entries(DefaultBoundHashOperations.java:102)
	org.springframework.session.data.redis.RedisOperationsSessionRepository.getSession(RedisOperationsSessionRepository.java:432)
	org.springframework.session.data.redis.RedisOperationsSessionRepository.getSession(RedisOperationsSessionRepository.java:402)
	org.springframework.session.data.redis.RedisOperationsSessionRepository.getSession(RedisOperationsSessionRepository.java:245)
	org.springframework.session.web.http.SessionRepositoryFilter$SessionRepositoryRequestWrapper.getSession(SessionRepositoryFilter.java:327)
	org.springframework.session.web.http.SessionRepositoryFilter$SessionRepositoryRequestWrapper.getSession(SessionRepositoryFilter.java:344)
	org.springframework.session.web.http.SessionRepositoryFilter$SessionRepositoryRequestWrapper.getSession(SessionRepositoryFilter.java:217)
	javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServletRequestWrapper.java:231)
	javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServletRequestWrapper.java:231)
	org.springframework.web.util.WebUtils.getSessionId(WebUtils.java:288)
	org.springframework.web.servlet.FrameworkServlet.publishRequestHandledEvent(FrameworkServlet.java:1077)
	org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1005)
	org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:861)
	javax.servlet.http.HttpServlet.service(HttpServlet.java:622)
	org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
	javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
	org.mycompany.identity.idp.service.security.GrantedAuthorityEntitlements.doFilter(GrantedAuthorityEntitlements.java:110)
	org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)
	org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:116)
	org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
	org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
	org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:116)
	org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
	org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
	org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
	org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:208)
	org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:185)
	org.apache.cxf.fediz.service.idp.STSPortFilter.doFilter(STSPortFilter.java:74)
	org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:101)
	org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:208)
	org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)
	org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
	org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)
	org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:101)
	org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:101)
	org.springframework.session.web.http.SessionRepositoryFilter.doFilterInternal(SessionRepositoryFilter.java:167)
	org.springframework.session.web.http.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:80)
	org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:101)
	org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:101)
	org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:101)
	org.springframework.boot.web.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:108)
	org.springframework.boot.web.support.ErrorPageFilter.forwardToErrorPage(ErrorPageFilter.java:183)
	org.springframework.boot.web.support.ErrorPageFilter.handleException(ErrorPageFilter.java:166)
	org.springframework.boot.web.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:130)
	org.springframework.boot.web.support.ErrorPageFilter.access$000(ErrorPageFilter.java:59)
	org.springframework.boot.web.support.ErrorPageFilter$1.doFilterInternal(ErrorPageFilter.java:90)
	org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	org.springframework.boot.web.support.ErrorPageFilter.doFilter(ErrorPageFilter.java

java.io.InvalidClassException: org.springframework.security.web.savedrequest.DefaultSavedRequest; local class incompatible: stream classdesc serialVersionUID = 8713433498670862907, local class serialVersionUID = 6412739018021506926
	java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:616)
	java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1843)
	java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713)
	java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2000)
	java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
	java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
	org.springframework.core.serializer.DefaultDeserializer.deserialize(DefaultDeserializer.java:70)
	org.springframework.core.serializer.support.DeserializingConverter.convert(DeserializingConverter.java:73)
	org.springframework.core.serializer.support.DeserializingConverter.convert(DeserializingConverter.java:36)
	org.springframework.data.redis.serializer.JdkSerializationRedisSerializer.deserialize(JdkSerializationRedisSerializer.java:80)
	org.springframework.data.redis.core.AbstractOperations.deserializeHashValue(AbstractOperations.java:338)
	org.springframework.data.redis.core.AbstractOperations.deserializeHashMap(AbstractOperations.java:282)
	org.springframework.data.redis.core.DefaultHashOperations.entries(DefaultHashOperations.java:227)
	org.springframework.data.redis.core.DefaultBoundHashOperations.entries(DefaultBoundHashOperations.java:102)
	org.springframework.session.data.redis.RedisOperationsSessionRepository.getSession(RedisOperationsSessionRepository.java:432)
	org.springframework.session.data.redis.RedisOperationsSessionRepository.getSession(RedisOperationsSessionRepository.java:402)
	org.springframework.session.data.redis.RedisOperationsSessionRepository.getSession(RedisOperationsSessionRepository.java:245)
	org.springframework.session.web.http.SessionRepositoryFilter$SessionRepositoryRequestWrapper.getSession(SessionRepositoryFilter.java:327)
	org.springframework.session.web.http.SessionRepositoryFilter$SessionRepositoryRequestWrapper.getSession(SessionRepositoryFilter.java:344)
	org.springframework.session.web.http.SessionRepositoryFilter$SessionRepositoryRequestWrapper.getSession(SessionRepositoryFilter.java:217)
	javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServletRequestWrapper.java:231)
	javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServletRequestWrapper.java:231)
	org.springframework.web.util.WebUtils.getSessionId(WebUtils.java:288)
	org.springframework.web.servlet.FrameworkServlet.publishRequestHandledEvent(FrameworkServlet.java:1077)
	org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1005)
	org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:861)
	javax.servlet.http.HttpServlet.service(HttpServlet.java:622)
	org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
	javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
	org.mycompany.identity.idp.service.security.GrantedAuthorityEntitlements.doFilter(GrantedAuthorityEntitlements.java:110)
	org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)
	org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:116)
	org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
	org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
	org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:116)
	org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
	org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
	org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
	org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:208)
	org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:185)
	org.apache.cxf.fediz.service.idp.STSPortFilter.doFilter(STSPortFilter.java:74)
	org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:101)
	org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:208)
	org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)
	org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
	org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)
	org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:101)
	org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:101)
	org.springframework.session.web.http.SessionRepositoryFilter.doFilterInternal(SessionRepositoryFilter.java:167)
	org.springframework.session.web.http.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:80)
	org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:101)
	org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:101)
	org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:101)
	org.springframework.boot.web.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:108)
	org.springframework.boot.web.support.ErrorPageFilter.forwardToErrorPage(ErrorPageFilter.java:183)
	org.springframework.boot.web.support.ErrorPageFilter.handleException(ErrorPageFilter.java:166)
	org.springframework.boot.web.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:130)
	org.springframework.boot.web.support.ErrorPageFilter.access$000(ErrorPageFilter.java:59)
	org.springframework.boot.web.support.ErrorPageFilter$1.doFilterInternal(ErrorPageFilter.java:90)
	org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	org.springframework.boot.web.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:108)

[OrangeDog]

Have there been any thoughts/progress on this? Without consistent means to (de)serialize core Spring Security objects, upgrades become very difficult.

As well as Session storage, the same applies to Cache (e.g. used by Spring Security ACL) and OAuth implementations. Not even a SimpleGrantedAuthority (which is little more than a String) can be serialised across minor versions at the moment.

There's a big difference between "we have avoided attempting to maintain a serializability contract between versions" and the current "we actively break all compatibility every version".

JDK serialisation need not be the solution (though many people are already stuck there). If e.g. Jackson were able to serialise most classes that would go a long way, but the majority are not designed as beans.

[eleftherias]

In order to do something like this we would need tests that ensure the serialization stays the same between versions.

For each class, we would need a file with the serialized output and a test that reads the file to produce the present-day deserialized object.

We would also need tests to ensure the present-day object can be serialized and then deserialized.

We are not actively working on this at the moment, however, pull requests are welcome.

[OrangeDog]

For each class, we would need a file with the serialized output and a test

No you don't. Again, we're not asking that you ensure it's always compatible, only that you stop deliberately making every minor version incompatible. All you have to do is stop changing the serialVersionUid on everything. Remove them and put it back how it was, or even better leave them as they are and only change them for each individual class if that class is changed to be incompatible.

[eleftherias]

@OrangeDog In order to only change the serialVersionUid for each individual class if that class is changed to be incompatible, we would need to know when the class is incompatible.
That's what the tests are for. They would ensure that the serialization remains compatible, and notify us when it is no longer compatible.

[OrangeDog]

@eleftherias most people know when a class is changed to be incompatible because they changed it to be incompatible. You also have code review in case they forget, and even if it's missed the worst thing that happens is you get a slightly different exception.

[lrozenblyum]

The same affects HA update for environments using Tomcat session clustering.

[Matthew-P-T]

So, I'm getting this error. I'm a beginner and I don't know what to do about it. I get that serialisation is the saving of the (state of the?) class somewhere and since upgrading I assume it is trying to read it back from somewhere and is failing because the numbers are different. Where is it trying to read it back from? Should I care what is saved wherever it is saved? What can I do about this please?