启动 Apache Atlas 时出错
1. 问题描述
启动Apache Atlas时,报错信息如下
Caused by: org.janusgraph.diskstorage.PermanentBackendException: Read 1 locks with our rid 97- 99- 49- 50- 49- 48- 50- 54- 49- 51- 56- 49- 54- 45-106-102-112- 49 but mismatched timestamps; no lock column contained our timestamp (2020-10-12T02:09:41.393999800Z)
at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.checkSeniority(ConsistentKeyLocker.java:528)
at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.checkSingleLock(ConsistentKeyLocker.java:454)
at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.checkSingleLock(ConsistentKeyLocker.java:118)
at org.janusgraph.diskstorage.locking.AbstractLocker.checkLocks(AbstractLocker.java:351)
... 107 more
2. 解决办法
JDK版本问题,JDK11的时间精度不兼容此版本JanusGraph默认的时间精度,JAVA_HOME替换成JDK8启动正常(其他JDK版本也可,JDK8亲测可以)
3. 详细原因
从Java 11开始
Instant.now()
的默认时间精度从milis(毫秒)提高到nano(纳秒),这使得JanusGraphTimestampProviders
无论实际配置了什么枚举值,始终使用NANO精度。而cassandra仍然使用MICRO精度,使得JanusGraph时间戳不兼容。
- 不同JDK版本下运行测试运行
public static void main(String[] args) {
Instant now = Instant.now();
System.out.println("default: " + now);
// JDK8 => out 默认为MILLTS精度
//default: 2020-10-12T08:09:45.609Z
// JDK11 => out 默认为NANOS精度
// default: 2020-10-12T08:05:21.304999800Z
}
- 以下参考 JanusGraph-issues https://github.com/JanusGraph/janusgraph/issues/1979
Starting from Java 11 the default precision of
Instant.now()
has been raised for some OSes from milis to nano. See similar issue: h2database/h2database#1178 This makes JanusGraphTimestampProviders
always uses NANO precision no matter what enum value is actually configured. Still cassandra uses MICRO precision which makes Janus timestamps incompatible - see stacktrace below. Fixed version ofTimestampProviders
:
MICRO {
@Override
public Instant getTime() {
// original => return Instant.now();
return Instant.now().truncatedTo(ChronoUnit.MICROS);
}
@Override
public Instant getTime(long sinceEpoch) {
return Instant.ofEpochSecond(0, (sinceEpoch * 1000L));
}
@Override
public ChronoUnit getUnit() {
return ChronoUnit.MICROS;
}
@Override
public long getTime(Instant timestamp) {
return timestamp.getEpochSecond() * 1000000L + timestamp.getNano()/1000;
}
},
MILLI {
@Override
public Instant getTime() {
// original => return Instant.now();
return Instant.now().truncatedTo(ChronoUnit.MILLIS);
}
@Override
public Instant getTime(long sinceEpoch) {
return Instant.ofEpochMilli(sinceEpoch);
}
@Override
public ChronoUnit getUnit() {
return ChronoUnit.MILLIS;
}
@Override
public long getTime(Instant timestamp) {
return timestamp.getEpochSecond() * 1000 + timestamp.getNano() / 1000000;
}
};
Note lines: return Instant.now().truncatedTo(ChronoUnit.MICROS); return Instant.now().truncatedTo(ChronoUnit.MILLIS); The above two lines fix the issue.
4. 详细错误堆栈信息如下
2020-10-12 10:09:51,410 ERROR - [main:] ~ Could not commit transaction [1] due to storage exception in system-commit (StandardJanusGraph:724)
org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56)
at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91)
at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139)
at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196)
at org.janusgraph.diskstorage.BackendTransaction.commit(BackendTransaction.java:150)
at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:721)
at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1425)
at org.janusgraph.graphdb.database.management.ManagementSystem.commit(ManagementSystem.java:195)
at org.apache.atlas.repository.graphdb.janus.AtlasJanusGraphManagement.commit(AtlasJanusGraphManagement.java:129)
at org.apache.atlas.repository.graph.GraphBackedSearchIndexer.commit(GraphBackedSearchIndexer.java:943)
at org.apache.atlas.repository.graph.GraphBackedSearchIndexer.initialize(GraphBackedSearchIndexer.java:363)
at org.apache.atlas.repository.graph.GraphBackedSearchIndexer.<init>(GraphBackedSearchIndexer.java:129)
at org.apache.atlas.repository.graph.GraphBackedSearchIndexer.<init>(GraphBackedSearchIndexer.java:116)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:142)
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:122)
at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:271)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1198)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1100)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:511)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:481)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:208)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.addCandidateEntry(DefaultListableBeanFactory.java:1314)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:1280)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveMultipleBeans(DefaultListableBeanFactory.java:1178)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1094)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1064)
at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:835)
at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:741)
at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:189)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1198)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1100)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:511)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:481)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:208)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1136)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1064)
at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:835)
at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:741)
at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:189)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1198)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1100)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:511)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:481)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:208)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.addCandidateEntry(DefaultListableBeanFactory.java:1314)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:1280)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveMultipleBeans(DefaultListableBeanFactory.java:1178)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1094)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1064)
at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:835)
at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:741)
at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:189)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1198)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1100)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:511)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:481)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:761)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543)
at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:443)
at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:325)
at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:107)
at org.apache.atlas.web.setup.KerberosAwareListener.contextInitialized(KerberosAwareListener.java:31)
at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:843)
at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:533)
at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:816)
at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:345)
at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1404)
at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1366)
at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:778)
at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:262)
at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:520)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
at org.eclipse.jetty.server.Server.start(Server.java:422)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:105)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
at org.eclipse.jetty.server.Server.doStart(Server.java:389)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.apache.atlas.web.service.EmbeddedServer.start(EmbeddedServer.java:98)
at org.apache.atlas.Atlas.main(Atlas.java:133)
Caused by: org.janusgraph.diskstorage.locking.PermanentLockingException: Permanent locking failure
at org.janusgraph.diskstorage.locking.AbstractLocker.checkLocks(AbstractLocker.java:359)
at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.checkAllLocks(ExpectedValueCheckingTransaction.java:175)
at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.prepareForMutations(ExpectedValueCheckingTransaction.java:154)
at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:72)
at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94)
at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91)
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54)
... 100 more
Caused by: org.janusgraph.diskstorage.PermanentBackendException: Read 1 locks with our rid 97- 99- 49- 50- 49- 48- 50- 54- 49- 51- 56- 49- 54- 45-106-102-112- 49 but mismatched timestamps; no lock column contained our timestamp (2020-10-12T02:09:41.393999800Z)
at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.checkSeniority(ConsistentKeyLocker.java:528)
at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.checkSingleLock(ConsistentKeyLocker.java:454)
at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.checkSingleLock(ConsistentKeyLocker.java:118)
at org.janusgraph.diskstorage.locking.AbstractLocker.checkLocks(AbstractLocker.java:351)
... 107 more
备注:
刚开始接触数据中台,需要搭建相关开源项目,现在在排坑阶段,搭建过程中如遇问题可以私信讨论~
推荐阅读
-
在IntelliJ IDEA里的Tomcat中尝试启动web项目时出现错误:调用manageApp方法时,遇到org.apache.tomcat.util.modeler.BaseModelMBean invoke 出现异常
-
启动 Apache Atlas 时出错
-
尝试同时启动两个或多个 Spring 批处理作业时出错:ORA-08177:无法序列化此事务的访问
-
对话NGC蔡岩:从机制创新到价值沉淀,解析DeFi产品开发逻辑 |链捕手 - 真正的DeFi产品首先要有足够的安全性和稳定性,如果能在此基础上有一些功能创新,那就非常好了。像 Uniswap 这样逐渐成为 DeFi 基础架构的产品,可遇而不可求。 链式捕手:固定利率协议之前关注度比较高,但观察下来发现,大部分协议还是类似于传统金融CDO(抵押债务凭证)的玩法,风险系数很高,您如何理解这块业务的价值和风险? 蔡岩:确实有些定息协议类似CDO玩法,背后绑定一个债券,但并不是所有的定息协议都是这样的玩法,像这种CDO玩法的主要代表项目是88mph,背后绑定的是Aave、Compoud这样的借贷协议,在此基础上做定息和浮息债券;像APWine,背后同样是Aave,它会发行期货收益代币来锁定你的收益;Notional本身是做借贷市场的,在此基础上做定息协议。 非 CDO 的玩法,比如 Horizon,更像是一个利率撮合器,背后需要用户通过拍卖产生更合适的目标收益率;像 Saffron、BarnBridge 等是通过风险分级来定义不同的收益率。总的来说,创新还是挺多的。 价值层面是创新和想象力,因为在传统金融领域,比如银行做固定收益证券,或者评级机构给风险分级,这些业务都非常大,利润也很丰厚。而 DeFi 的对口业务给了类似业务很大的想象空间。尤其是固定利率协议的成熟产品不多,尝试各种微创新是很有意义的。 风险程度还是要具体到不同的玩法,比如,在 Aave、Compoud 等借贷协议的固定利率协议背后,如果这些借贷协议受到攻击,与之绑定的固定利率协议也会受损。 同样,如果自己做借贷市场,可能更需要更强的开发能力。再有,如果该程序的机制或参数设计不当,同样会导致协议运行不稳定,并可能造成大量用户清盘。 总的来说,风险在于固定利率协议的设计,这是一个非常复杂的过程,需要不断地尝试和出错。 链式捕捉器:刚刚提到背后是Aave/Compound的固定费率协议风险较大,您认为Aave最大的不确定性和创新点分在哪里? 蔡岩:其实爱钱进一直被认为是走在行业前列的项目,他们的迭代速度非常快,比如率先尝试闪贷、推出新的经济激励模式、推出目前业内首个安全模块、尝试L2解决方案等等。 而在主要的借贷业务上,他们又十分谨慎,比如在抵押率、清算系数等风险参数的设计上相对于其他借贷协议较为保守,并不会存在为了吸引更多借贷资金而降低风险的要求。 与许多 DeFi 项目一样,即使 Aave 进行了多次审计,也无法保证不存在漏洞。前段时间,Aave 刚进入 V2 阶段时,白帽黑客就指出了某个漏洞。 之前的创新点可能是闪电借贷,这是当时业内独一无二的新产品功能,也为 Aave 带来了不少收益。当然,也有人批评闪电贷只能方便黑客实现资金效益的最大化,但工具本身并没有错,未来闪电贷肯定会有更多的应用场景。 其次是安全模块的设计,这有点像项目本身的储备金库,保障项目的安全性,这也是爱维开创的先河。说实话,目前大多数项目都没有做到代币模式的良性或正向运营,也做不到像Aave一样的安全模块,这是一个不小的门槛。 Chaincatcher从某种程度上来说,挖矿模式是DeFi财富效应的根本支撑,但Aave的CEO却说挖矿机制带来的动力是不可持续的,您怎么看这个观点? 蔡岩:"挖矿机制 "不可能失效,因为它是一种激励机制,或者说是项目冷启动的一种方式。但流动性开采亚博体育手机客户端不会一直高涨。比如去年11月的流行性挖矿高APY持续了一两个月就崩盘了,导致DeFi市场大幅回调。 Aave、Uniswap、Synthetix等项目真正爆发进入市值前15名也是在今年2月,我更倾向于这是头部DeFi长期价值的体现。虽然大家都喜欢抢高APY的矿机,但我个人很少参与挖矿,所以我并不觉得流动性挖矿是DeFi的基本面支撑。
-
启动 vue 项目安装依赖项时出错的解决方案