Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mysql数据用canal同步到doris,无法落库到doris #5353

Open
lyj-refuel opened this issue Dec 14, 2024 · 1 comment
Open

mysql数据用canal同步到doris,无法落库到doris #5353

lyj-refuel opened this issue Dec 14, 2024 · 1 comment

Comments

@lyj-refuel
Copy link

lyj-refuel commented Dec 14, 2024

Question

场景:mysql 5.5的数据同步到doris,canal-adapter/logs/adapter/目录的的 日志可以捕获到binlog信息,但是写入不到doris,canal用的1.1.8,
application.yml 配置信息:

  outerAdapters:
  - name: logger
    key: mysql1
    properties:
      jdbc.driverClassName: com.mysql.jdbc.Driver
      jdbc.url: jdbc:mysql://192.168.7.161:9030/test?useUnicode=true
      jdbc.username: root
      jdbc.password: 

rdb/xx.yml

dataSourceKey: defaultDS
destination: example
groupId: g1
outerAdapterKey: mysql1
concurrent: true
dbMapping:
database: v3-all
table: ebay_order
targetTable: ebay_order
targetPk:
id: ebay_id
mapAll: true
commitBatch: 10 # 批量提交的大小

@lyj-refuel
Copy link
Author

lyj-refuel commented Dec 14, 2024

2024-12-14 14:37:32.967 [pool-8-thread-1] INFO c.a.o.canal.client.adapter.logger.LoggerAdapterExample - DML: {"data":[{"ebay_id":392595015,"ebay_ordersn":"cn1537622598zyjp5486052111978","ebay_orderqk":null,"ebay_paystatus":"0","recordnumber":"5486052905078","ebay_tid":"5486051905078","ebay_ptid":"","ebay_orderid":"","ebay_createdtime":0,"ebay_paidtime":1734141492,"ebay_userid":"Кошевой Дмитрий Павлович","ebay_username":"Кошевой Дмитрий Павлович","ebay_usermail":"","ebay_street":"г. Благовещенск, ул. 50 лет Октября, 15","ebay_street1":"г. Благовещенск, ул. 50 лет Октября, 15","ebay_city":"Благовещенск г","ebay_state":"Амурская обл","ebay_couny":"RU","ebay_countryname":"俄罗斯","ebay_postcode":"67004","ebay_phone":"963814314","ebay_currency":"USD","ebay_total":1.7,"ebay_status":1723,"ebay_user":"system","ebay_addtime":1734145359,"ebay_shipfee":"0.790000","ebay_combine":"0","market":"韦建秋","ebay_account":"cn15376228zyjp","ebay_note":"","ebay_noteb":":2025-02-27 11:02:39","is_reg":0,"ordertype":"aliexpress","status":"0","mailstatus":0,"templateid":"0","postive":"0","ebay_carrier":"速卖通线上无忧-简易带电","ebay_carrierstyle":"","ebay_warehouse":"234","ebay_markettime":0,"ebay_tracknumber":"","ebay_site":"","location":null,"eBayPaymentStatus":null,"PayPalEmailAddress":"","ShippedTime":"0","RefundAmount":0.0,"resendreason":null,"refundreason":null,"resendtime":null,"refundtime":null,"canceltime":null,"cancelreason":null,"ebay_feedback":0,"ebay_sdsn":null,"isprint":0,"ebay_ordertype":"0","profitstatus":0,"orderweight":0.024,"orderweight2":0.0,"ordershipfee":0.0,"ordercopst":1.0,"scantime":0,"accountid":22196,"ishide":1822,"packingtype":"0","packinguser":null,"packagingstaff":null,"order_no":"0","ebay_phone1":"+7-9638141314","ismodifive":0,"totalprofit":0.0,"ebay_case":0,"paypal_case":0,"moneyback":0,"moneyback_total":0.0,"orders_pcase_time":0,"orders_ecase_time":-1,"erp_op_id":1,"Custom_Paypalemailaddress":"{"firmpack":0,"firmpack_assist":0}","pxorderid":"","pxordertime":null,"cky_orderid":null,"cky_item":null,"ebay_company":null,"zencartid":"ru1763105224yyxae","updateprofittime":0,"smturl":null}],"database":"v3-all","destination":"example","es":1734147071000,"groupId":"g1","isDdl":false,"old":[{"ebay_status":1722}],"pkNames":["ebay_id"],"sql":"","table":"ebay_order","ts":1734158252955,"type":"UPDATE"}
2024-12-14 14:37:32.973 [pool-8-thread-1] ERROR c.a.otter.canal.adapter.launcher.loader.AdapterProcessor - java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.RuntimeException: Target column: id not matched
java.lang.RuntimeException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.RuntimeException: Target column: id not matched
at com.alibaba.otter.canal.client.adapter.rdb.RdbAdapter.sync(RdbAdapter.java:157)
at com.alibaba.otter.canal.client.adapter.ProxyOuterAdapter.sync(ProxyOuterAdapter.java:42)
at com.alibaba.otter.canal.adapter.launcher.loader.AdapterProcessor.batchSync(AdapterProcessor.java:142)
at com.alibaba.otter.canal.adapter.launcher.loader.AdapterProcessor.lambda$null$1(AdapterProcessor.java:100)
at java.base/java.util.concurrent.CopyOnWriteArrayList.forEach(CopyOnWriteArrayList.java:807)
at com.alibaba.otter.canal.adapter.launcher.loader.AdapterProcessor.lambda$null$2(AdapterProcessor.java:97)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.RuntimeException: Target column: id not matched
at com.alibaba.otter.canal.client.adapter.rdb.service.RdbSyncService.lambda$sync$2(RdbSyncService.java:135)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
at com.alibaba.otter.canal.client.adapter.rdb.service.RdbSyncService.sync(RdbSyncService.java:131)
at com.alibaba.otter.canal.client.adapter.rdb.service.RdbSyncService.sync(RdbSyncService.java:155)
at com.alibaba.otter.canal.client.adapter.rdb.RdbAdapter.sync(RdbAdapter.java:153)
... 9 common frames omitted

报错说没有id not matched,我设置有主键,doris表里也有主键

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant