incubator-doris.git
11 hours agoFix base compaction bug. (#421) master
lichaoyong [Wed, 12 Dec 2018 12:44:32 +0000 (20:44 +0800)] 
Fix base compaction bug. (#421)

Base compaction may choose tablet which has missed versions now.
After compaction, it will failed to check tablet integrity by versions and cored dump.
Ignore this tablet when find tablet to base_compaction.

12 hours agoAdd Md5Digest to util (#420)
ZHAO Chun [Wed, 12 Dec 2018 12:06:35 +0000 (20:06 +0800)] 
Add Md5Digest to util (#420)

20 hours agoFix compaction and ingestion core (#417)
lichaoyong [Wed, 12 Dec 2018 03:30:06 +0000 (11:30 +0800)] 
Fix compaction and ingestion core (#417)

Error occurs when reading data by compaction and ingestion.
Under the circumstance, the two operation should stop and return error.

38 hours agoFix compile error in run-fe-ut.sh (#415)
lide [Tue, 11 Dec 2018 09:46:13 +0000 (17:46 +0800)] 
Fix compile error in run-fe-ut.sh (#415)

40 hours agoFix compile failure in RLTaskTxnCommitAttachment (#414)
EmmyMiao87 [Tue, 11 Dec 2018 08:00:07 +0000 (16:00 +0800)] 
Fix compile failure in RLTaskTxnCommitAttachment (#414)

40 hours agoAdd JDK8 in the docker image (#413)
tangxiaoqing [Tue, 11 Dec 2018 07:42:07 +0000 (15:42 +0800)] 
Add JDK8 in the docker image (#413)

42 hours agoFix compile failure in GlobalTransactionMgrTest (#412)
EmmyMiao87 [Tue, 11 Dec 2018 05:53:38 +0000 (13:53 +0800)] 
Fix compile failure in GlobalTransactionMgrTest (#412)

45 hours agoAdd a frontend interface for committing RoutineLoadTask (#368)
EmmyMiao87 [Tue, 11 Dec 2018 03:06:25 +0000 (11:06 +0800)] 
Add a frontend interface for committing RoutineLoadTask (#368)

1. add a needSchedulerTasksQueue in LoadManager: the RoutineLoadTaskScheduler will poll task from this queue and schedule task.
2. add a frontend interface named rlTaskCommit: commit txn, update offset and renew a task for the same partitions
3. add extra property in transaction state: in rlTaskCommit, extra property which looks like {"job_id": xxx, "progress": xxx}
When fe initialize routine load job meta from logs, all of txn state which related to routine load job will be used for initializing progress of job.

Add a TxnStateChangeListener interface for transaction
1. onCommitted , onAborted, beforeAborted will be called by different type of txn
2. RoutineLoadJob will update job progress and renew a task when onCommitted callback
3. Add TxnStateChangeListener into TransactionState
4. set transactionState to committed will call onCommitted callback if callback is not null
5. set transactionState to aborted will call beforeAborted and onAborted
6. beforeAborted in RoutineLoadJob will check if there is related task when TxnStatusChangeReason is TIMEOUT. It will prevent abort when there is a related task by throw TransactionException
7. Other reason of abort will not prevent abort. The onAborted will be call and job state will be change to paused

Change extra to TxnCommitAttachment in TLoadTxnCommitRequest
1. The KAFKA source of TTxnSourceType means that this is a routine load task commit. And the TRLTaskTxnCommitAttachment is the commitInfo of this task.
2. TRLTaskTxnCommitAttachment will be convert to RLTaskTxnCommitAttachment which include progress of this task, task id, numOfErrorData etc.

Add param TxnCommitAttachment into commitTransaction
1. The TxnCommitAttachment will be updated in commitTransaction

2 days agoClear client pool when heartbeat failed (#408)
Mingyu Chen [Mon, 10 Dec 2018 10:52:51 +0000 (18:52 +0800)] 
Clear client pool when heartbeat failed (#408)

When heartbeat failed, we should clear the connections cached
in client pool, or we will get broken connections from the pool.
Since we don't have the REOPEN logic(which may cause ugly code style),
a broken connection may cause a rpc blocked and failed.
So clear them all and recreate them when needed is a simple way to
resolve this problem.

We only clear connections in backend and broker pool.
No need to clear heartbeat pool because heartbeat is very frequent,
such the connections can be invalid automatically.

2 days agoRelease push_lock when schema change failed (#407)
lichaoyong [Mon, 10 Dec 2018 10:39:19 +0000 (18:39 +0800)] 
Release push_lock when schema change failed (#407)

Release push_lock when schema change failed

2 days agoRefactor heartbeat logic (#403)
Mingyu Chen [Mon, 10 Dec 2018 06:41:12 +0000 (14:41 +0800)] 
Refactor heartbeat logic (#403)

* Refactor heartbeat logic

Currently we only have Backend heartbeat. And without Frontend
or Broker heartbeat, we don't know the status of these nodes,
thus can't do failover logic in some cases.

1. Add Frontend and Broker heartbeat.
    Frontend heartbeat using BootstrapFinish http rest api
    Broker heartbeat using ping() rpc.
2. All heartbeats are managed in HeartbeatMgr.
3. Rename BrokerAddress to FsBroker.

2 days agoImprove build scripts (#404)
lide [Mon, 10 Dec 2018 05:56:09 +0000 (13:56 +0800)] 
Improve build scripts (#404)

6 days agoFix bug that null value will be loaded to non-nullable column (#401)
Mingyu Chen [Thu, 6 Dec 2018 11:55:55 +0000 (19:55 +0800)] 
Fix bug that null value will be loaded to non-nullable column (#401)

* Fix bug that null value will be loaded to non-nullable column

* Optimize performance

6 days agoFix bug that ColumnType is no longer used (#400)
Mingyu Chen [Thu, 6 Dec 2018 11:19:29 +0000 (19:19 +0800)] 
Fix bug that ColumnType is no longer used (#400)

6 days agoSupport Colocate Join (#245) (#246)
kangkaisen [Thu, 6 Dec 2018 10:59:17 +0000 (18:59 +0800)] 
Support Colocate Join (#245) (#246)

* Support colocate join

Colocate join means two table are distributed by the columns being joined,
then we can join them locally on each backend.

Colocate join no data movement and has more concurrency.

6 days agoFix bug that null value will be loaded to non-nullable column (#397)
Mingyu Chen [Thu, 6 Dec 2018 02:09:34 +0000 (10:09 +0800)] 
Fix bug that null value will be loaded to non-nullable column (#397)

7 days agoAdd 'fileNameOnly' param to broker's listPath method (#394)
Mingyu Chen [Wed, 5 Dec 2018 12:49:31 +0000 (20:49 +0800)] 
Add 'fileNameOnly' param to broker's listPath method (#394)

7 days agoRevert 'Support 'NO_BACKSLASH_ESCAPES' sql_mode (#392)
Mingyu Chen [Wed, 5 Dec 2018 12:23:34 +0000 (20:23 +0800)] 
Revert 'Support 'NO_BACKSLASH_ESCAPES' sql_mode (#392)

7 days agoFix building Docker image error (#390)
tangxiaoqing [Wed, 5 Dec 2018 10:27:50 +0000 (18:27 +0800)] 
Fix building Docker image error (#390)

8 days agoFix compile failure in ScanNode (#384)
chenhao [Tue, 4 Dec 2018 08:51:48 +0000 (16:51 +0800)] 
Fix compile failure in ScanNode (#384)

8 days agoSupport 'NO_BACKSLASH_ESCAPES' sql_mode (#382)
Mingyu Chen [Tue, 4 Dec 2018 03:33:04 +0000 (11:33 +0800)] 
Support 'NO_BACKSLASH_ESCAPES' sql_mode (#382)

8 days agoFix cast error in BrokerScanNode (#383)
chenhao [Tue, 4 Dec 2018 03:30:03 +0000 (11:30 +0800)] 
Fix cast error in BrokerScanNode (#383)

9 days agoUnify Slice code path (#380)
李超勇 [Mon, 3 Dec 2018 10:11:47 +0000 (18:11 +0800)] 
Unify Slice code path (#380)

9 days agoFix a sending signal error when starting Doris BE (#367)
Liu Xudong [Mon, 3 Dec 2018 07:38:33 +0000 (15:38 +0800)] 
Fix a sending signal error when starting Doris BE (#367)

Redirect output message of kill to /dev/null.

Co-Authored-By: chalsliu <45041955+chalsliu@users.noreply.github.com>
ISSUE #365

9 days agoRemove OLAP_LOG_DEBUG AND OLAP_LOG_TRACE log format (#378)
李超勇 [Mon, 3 Dec 2018 02:08:21 +0000 (10:08 +0800)] 
Remove OLAP_LOG_DEBUG AND OLAP_LOG_TRACE log format (#378)

Use VLOG(3) and VLOG(10) instead

9 days agoSupport _IP/_HOST in principal of kerberos (#373)
zhaidongbo [Mon, 3 Dec 2018 02:03:57 +0000 (10:03 +0800)] 
Support _IP/_HOST in principal of kerberos (#373)

11 days agoSupport TRUNCATE TABLE stmt (#377)
Mingyu Chen [Sat, 1 Dec 2018 13:18:27 +0000 (21:18 +0800)] 
Support TRUNCATE TABLE stmt (#377)

* Support TRUNCATE TABLE stmt

User can use TRUNCATE TABLE stmt to empties a table
or partitions completely.
Unlike DELETE, it will drop the tablets directly, and
without any performance impact.

* Fix bugs that new partition should use new ID

* Use equals() to compare Integer

* Fix compile bug

* Fix bug on single range parititon

* Check table's state again after creating partition

11 days agoRemove OLAP_LOG_FATAL log format. Use LOG(FATAL) instead (#376)
李超勇 [Sat, 1 Dec 2018 11:26:08 +0000 (19:26 +0800)] 
Remove OLAP_LOG_FATAL log format. Use LOG(FATAL) instead (#376)

12 days agoRemove OLAP_LOG_INFO log format. Use LOG(INFO) instead (#372)
李超勇 [Fri, 30 Nov 2018 12:59:40 +0000 (20:59 +0800)] 
Remove OLAP_LOG_INFO log format. Use LOG(INFO) instead (#372)

12 days agoRevert "Remove OLAP_LOG_INFO log format. Use LOG(INFO) instead (#370)" (#371)
ZHAO Chun [Fri, 30 Nov 2018 12:56:51 +0000 (20:56 +0800)] 
Revert "Remove OLAP_LOG_INFO log format. Use LOG(INFO) instead (#370)" (#371)

This reverts commit a816925776de06dc7503ea7429802cad9042d0e4.

12 days agoRemove OLAP_LOG_INFO log format. Use LOG(INFO) instead (#370)
李超勇 [Fri, 30 Nov 2018 12:36:58 +0000 (20:36 +0800)] 
Remove OLAP_LOG_INFO log format. Use LOG(INFO) instead (#370)

* Remove unused row-oriented format flags

* Remove unused row-oriented format flags

* Remove OLAP_LOG_INFO log format. Use LOG(INFO) instead

12 days agoSubsititue ColumnType to Type (#366)
ZHAO Chun [Fri, 30 Nov 2018 08:30:30 +0000 (16:30 +0800)] 
Subsititue ColumnType to Type (#366)

* Subsititue ColumnType to Type

13 days agoRename Rowset to SegmentGroup (#364)
kangpinghuang [Thu, 29 Nov 2018 09:30:41 +0000 (17:30 +0800)] 
Rename Rowset to SegmentGroup (#364)

* Rename Rowset to SegmentGroup

* Modify protobuf related rowset to SegmentGroup

13 days agoFix stream load failure when target table contains HLL and insert failure when it...
chenhao [Thu, 29 Nov 2018 07:40:04 +0000 (15:40 +0800)] 
Fix stream load failure when target table contains HLL and insert failure when it contains subquery (#359)

2 weeks agoFix snapshot's making header bug (#362)
kangpinghuang [Wed, 28 Nov 2018 10:58:21 +0000 (18:58 +0800)] 
Fix snapshot's making header bug (#362)

2 weeks agoUbuntu llvm compile (#361)
lide [Wed, 28 Nov 2018 07:22:00 +0000 (15:22 +0800)] 
Ubuntu llvm compile (#361)

2 weeks agoFix insert error when it contains HLL (#358)
chenhao [Tue, 27 Nov 2018 09:10:41 +0000 (17:10 +0800)] 
Fix insert error when it contains HLL (#358)

2 weeks agoRemove unused row-oriented format flags (#357)
李超勇 [Tue, 27 Nov 2018 08:34:22 +0000 (16:34 +0800)] 
Remove unused row-oriented format flags (#357)

2 weeks agoFix cast error in StreamLoadScanNodeush (#356)
chenhao [Tue, 27 Nov 2018 08:03:33 +0000 (16:03 +0800)] 
Fix cast error in StreamLoadScanNodeush (#356)

2 weeks agoAdd connection id to CurrentQueryStatisticsProcDir (#355)
chenhao [Tue, 27 Nov 2018 06:28:39 +0000 (14:28 +0800)] 
Add connection id to CurrentQueryStatisticsProcDir (#355)

2 weeks agoAvoid 'No more data to read' error when handling stream load RPC (#354)
Mingyu Chen [Tue, 27 Nov 2018 06:18:41 +0000 (14:18 +0800)] 
Avoid 'No more data to read' error when handling stream load RPC (#354)

* Avoid 'No more data to read' error when handling stream load rpc

1. Catch throwable of all stream load rpc.
2. Avoid setting null string as error msg of rpc result status.

* Change setError_msgs to addToError_msgs

2 weeks agoOptimize the publish logic of streaming load (#350)
Mingyu Chen [Mon, 26 Nov 2018 11:01:50 +0000 (19:01 +0800)] 
Optimize the publish logic of streaming load (#350)

1. Only collect all error replicas if publish task is timeout.
2. Add 2 metrics to monitor the success of failure of txn.
3. Change publish timeout to Config.load_straggler_wait_second

2 weeks agoImprove the Backend's disk info report performance (#349)
Mingyu Chen [Mon, 26 Nov 2018 05:42:58 +0000 (13:42 +0800)] 
Improve the Backend's disk info report performance (#349)

1. Use the data size save in tablet's header to calculate the disk used capacity.
2. Decrease the default interval of disk and tablet report, from 10 min to 1 min.

2 weeks agoFix wrong query result when column value is Null (#344)
kangkaisen [Mon, 26 Nov 2018 05:36:23 +0000 (13:36 +0800)] 
Fix wrong query result when column value is Null (#344)

2 weeks agoUpdate README.md (#347)
lide [Fri, 23 Nov 2018 07:40:07 +0000 (15:40 +0800)] 
Update README.md (#347)

Move old content of READEME.md to the Overview of Wiki.
Add build guide and user cases to README.md.

2 weeks agoAdd distributor which schedule task to be fairly, for routine load job (#333)
EmmyMiao87 [Fri, 23 Nov 2018 02:35:10 +0000 (10:35 +0800)] 
Add distributor which schedule task to be fairly, for routine load job (#333)

Step1: updateBeIdTaskMaps, remove unavailable BE and add new alive BE
Step2: process timeout tasks, if a task has already been allocated to BE but not finished before DEFAULT_TASK_TIMEOUT, it will be discarded.
       At the same time, the partitions belong to old tasks will be allocated to a new task. The new task with a signature will be added in the queue of needSchedulerRoutineLoadTask.
Step3: process all needSchedulerRoutineLoadTasks, allocate task to BE. The task will be executed by BE.

2 weeks agoFix bug of using symbolic link dir as storage path (#340)
kangpinghuang [Fri, 23 Nov 2018 02:12:26 +0000 (10:12 +0800)] 
Fix bug of using symbolic link dir as storage path (#340)

* Fix bug of #307

There is a bug to use symbolic link directory as storage root path.
It is a problem that whether the path is canonical

In DownloadAction, checking fails by comparing canonical path with non-canonical path.
So fix the bug by convert all path to canonical path before comparison

2 weeks agoChange ByteBuffer to StorageByteBuffer in olap/byte_buffer.h (#341)
李超勇 [Thu, 22 Nov 2018 15:18:33 +0000 (23:18 +0800)] 
Change ByteBuffer to StorageByteBuffer in olap/byte_buffer.h (#341)

Class definition of ByteBuffer duplicates between olap/byte_buffer.h and util/byte_buffer.h.
All of the two classes has a function names as remaining().
Some place which want to call remaining() of util/byte_buffer.h is linked to the other remaining() function of olap/byte_buffer.h

2 weeks agoAdd special add_column_statistics method for linked_schema_change (#337)
kangkaisen [Thu, 22 Nov 2018 07:39:00 +0000 (15:39 +0800)] 
Add special add_column_statistics method for linked_schema_change (#337)

2 weeks agoFix a compile issue of DORIS_GCC_HOME (#339)
lide [Thu, 22 Nov 2018 04:32:14 +0000 (12:32 +0800)] 
Fix a compile issue of DORIS_GCC_HOME (#339)

2 weeks agoFix compile issue of thirdparty library (#338)
lide [Thu, 22 Nov 2018 03:15:42 +0000 (11:15 +0800)] 
Fix compile issue of thirdparty library (#338)

3 weeks agoModify partition's version name to what it means (#334)
Mingyu Chen [Wed, 21 Nov 2018 11:21:16 +0000 (19:21 +0800)] 
Modify partition's version name to what it means (#334)

* Modify partition's version name to what it means.

1. committedVersion(Hash) -> visibleVersion(Hash)
2. currentVersion(Hash) -> committedVersion(Hash)
3. add some comment to make the code more readable

* Check if editlog is null in CatalogIdGenerator
    To avoid unit test failure

3 weeks agoSupport for custom build toolchains (#330)
Liu Xudong [Wed, 21 Nov 2018 07:58:12 +0000 (15:58 +0800)] 
Support for custom build toolchains (#330)

* Support for custom build toolchains

ISSUE #297

Locate the header file path of DORIS_GCC_HOME, add them to the CPLUS_INCLUDE_PATH environment variable.
So clang can find it.

* Compatible with custom toolchain for clang

ISSUE #297

1. Find default header file paths of DORIS_GCC_HOME and set as enviroment variable CLANG_COMPATIBLE_FLAGS.
2. Explicitly add these paths to CLANG_INCLUDE_FLAGS.

3 weeks agoFix a compile issue (#331)
lide [Tue, 20 Nov 2018 12:06:36 +0000 (20:06 +0800)] 
Fix a compile issue (#331)

* Fix a compile issue

* Fix a compile issue

3 weeks agoChange PaloMetrics' name and Catalog's Id generator (#329)
Mingyu Chen [Tue, 20 Nov 2018 10:59:18 +0000 (18:59 +0800)] 
Change PaloMetrics' name and Catalog's Id generator (#329)

* Change PaloMetrics' name and Catalog's Id generator
1. Remove 'Palo' prefix of class Metric.
2. Add a new CatalogIdGenerator to replace the old AtomicLong, to avoid too many edit logs.
3. Add a new histogram to monitor write letency of edit log write.

* modify next id logic

* fix a bug that Metric is not init before using HISTO_EDIT_LOG_WRITE_LATENCY

* fix a problem

3 weeks agoAdd path info of replica in catalog (#327)
Mingyu Chen [Mon, 19 Nov 2018 09:42:46 +0000 (17:42 +0800)] 
Add path info of replica in catalog (#327)

Add path info of replica in catalog

Also fix a bug that when calling check_none_row_oriented_table,
store is null, it cannot be used to create table.
Instead, OLAPHeader can be used to get storage type information.

3 weeks agoFix fe.conf audit_log_moudles,sys_log_roll_mode,audit_log_roll_mode settings (#326)
lichao [Sat, 17 Nov 2018 14:18:19 +0000 (22:18 +0800)] 
Fix fe.conf audit_log_moudles,sys_log_roll_mode,audit_log_roll_mode settings (#326)

3 weeks agoChange log verbose level to vlog(3) (#325)
李超勇 [Fri, 16 Nov 2018 09:17:39 +0000 (17:17 +0800)] 
Change log verbose level to vlog(3) (#325)

* Transform row-oriented table to columnar-oriented table

* Transform row-oriented table to columnar-oriented table

* change log verbose level

3 weeks agoAdd docker dev environment (#301)
tangxiaoqing [Fri, 16 Nov 2018 08:35:08 +0000 (16:35 +0800)] 
Add docker dev environment (#301)

Add a docker based dev environment.
User can build Doris from source in docker container.

3 weeks agoTransform row-oriented table to columnar-oriented table (#311)
李超勇 [Fri, 16 Nov 2018 08:03:56 +0000 (16:03 +0800)] 
Transform row-oriented table to columnar-oriented table (#311)

3 weeks agoFix core local value UT failed (#324)
Zhao Chun [Fri, 16 Nov 2018 07:27:16 +0000 (15:27 +0800)] 
Fix core local value UT failed (#324)

Issue: #323

3 weeks agoFix fe.conf audit_log_moudles default value error (#321)
lichao [Fri, 16 Nov 2018 00:54:57 +0000 (08:54 +0800)] 
Fix fe.conf audit_log_moudles default value error (#321)

3 weeks agoAdd scheduler routine load job for stream load (#313)
EmmyMiao87 [Thu, 15 Nov 2018 13:04:22 +0000 (21:04 +0800)] 
Add scheduler routine load job for stream load (#313)

1. fetch need_scheduler routine load job
2. caculate current concurrent task number of job
3. divide kafka partition into tasks

3 weeks agoFix SHOW BACKENDS return ERROR (#320)
Zhao Chun [Thu, 15 Nov 2018 12:14:39 +0000 (20:14 +0800)] 
Fix SHOW BACKENDS return ERROR (#320)

In some case, errMsg in Backend maybe null. we change it when check it
nut null

Issue: #317

3 weeks agoFix lose of be's meta data bug (#318)
kangpinghuang [Thu, 15 Nov 2018 11:23:42 +0000 (19:23 +0800)] 
Fix lose of be's meta data bug (#318)

Modify rocksdb's WriteOptions.sync to true
to force sync WAL to disk every write operation

3 weeks agoReduce UT binary size (#314)
Zhao Chun [Thu, 15 Nov 2018 08:17:23 +0000 (16:17 +0800)] 
Reduce UT binary size (#314)

* Reduce UT binary size

Almost every module depend on ExecEnv, and ExecEnv contains all
singleton, which make UT binary contains all object files.

This patch seperate ExecEnv's initial and destory to anthor file to
avoid other file's dependence. And status.cc include debug_util.h which
depend tuple.h tuple_row.h, and I move get_stack_trace() to
stack_util.cpp to reduce status.cc's dependence.

I add USE_RTTI=1 to build rocksdb to avoid linking librocksdb.a

Issue: #292

* Update

3 weeks agoOptimize quota unit (#309)
kangpinghuang [Thu, 15 Nov 2018 06:03:52 +0000 (14:03 +0800)] 
Optimize quota unit (#309)

Originally, we can only set quota in bytes unit. This commit add quota unit:K/KB/M/MB/G/GB/T/TB/P/PB
for convenience.

4 weeks agoImprove build script (#305)
lide [Wed, 14 Nov 2018 06:10:46 +0000 (14:10 +0800)] 
Improve build script (#305)

4 weeks agoMake BE UT success (#304)
Zhao Chun [Tue, 13 Nov 2018 11:17:51 +0000 (19:17 +0800)] 
Make BE UT success (#304)

4 weeks agoFix code LICENSE for file modified from LevelDB. (#300)
李超勇 [Mon, 12 Nov 2018 08:09:40 +0000 (16:09 +0800)] 
Fix code LICENSE for file modified from LevelDB. (#300)

4 weeks agoFix failed cases in regression test (#299)
chenhao7253886 [Mon, 12 Nov 2018 03:15:39 +0000 (11:15 +0800)] 
Fix failed cases in regression test (#299)

4 weeks agoCorrect TxnKey comment (#295)
kangpinghuang [Sat, 10 Nov 2018 07:49:10 +0000 (15:49 +0800)] 
Correct TxnKey comment (#295)

4 weeks agoBe compatible with old RPC (#296)
Zhao Chun [Sat, 10 Nov 2018 07:46:45 +0000 (15:46 +0800)] 
Be compatible with old RPC (#296)

Add palo.PInternalService which can server old version palo's client.

Issue: #293

4 weeks agoFix some license (#290)
lide [Fri, 9 Nov 2018 06:30:09 +0000 (14:30 +0800)] 
Fix some license (#290)

4 weeks agoFix be build error (#289)
tangxiaoqing [Fri, 9 Nov 2018 02:39:03 +0000 (10:39 +0800)] 
Fix be build error (#289)

#288

5 weeks agoImprove cardinality, avgRowSize, numNodes stat info in OlapScanNode (#256)
kangkaisen [Wed, 7 Nov 2018 05:59:05 +0000 (13:59 +0800)] 
Improve cardinality, avgRowSize, numNodes stat info in OlapScanNode (#256)

Currently, the cardinality, avgRowSize, numNodes stat info in OlapScanNode is none, So the broadcastCost and partitionCost are both wrong and Doris couldn't auto choose a best join strategy.

So we should make the statistical information in OlapScanNode more precise.

5 weeks agoFix unit test failure (#286)
chenhao7253886 [Wed, 7 Nov 2018 04:53:11 +0000 (12:53 +0800)] 
Fix unit test failure (#286)

5 weeks agoFix BE can't be grayscale upgraded (#285)
chenhao7253886 [Wed, 7 Nov 2018 01:34:39 +0000 (09:34 +0800)] 
Fix BE can't be grayscale upgraded (#285)

5 weeks agoFix truncation error in CastExpr (#283)
chenhao7253886 [Tue, 6 Nov 2018 10:57:13 +0000 (18:57 +0800)] 
Fix truncation error in CastExpr (#283)

5 weeks agoSimplify constant Expr (#255)
kangkaisen [Tue, 6 Nov 2018 09:24:54 +0000 (17:24 +0800)] 
Simplify constant Expr (#255)

Simplify constant Expr could improve Partition Pruning. The examples for constant Expr Simplifing:

1 + 1 + 1 --> 3
date_add('2018-08-08', 1) --> 2018-08-09
year('2018-07-24')*12 + month('2018-07-24') -> 24223

5 weeks agoFix view missed parenthesis bug (#253)
kangkaisen [Tue, 6 Nov 2018 07:25:40 +0000 (15:25 +0800)] 
Fix view missed parenthesis bug (#253)

5 weeks agoSupport AnalyticExpr in View (#248)
kangkaisen [Mon, 5 Nov 2018 12:39:21 +0000 (20:39 +0800)] 
Support AnalyticExpr in View (#248)

5 weeks agoSupport NULLS LAST and NULLS FIRST syntax (#252)
kangkaisen [Mon, 5 Nov 2018 12:35:10 +0000 (20:35 +0800)] 
Support NULLS LAST and NULLS FIRST syntax (#252)

Allow User specify the null ordering

NULLS FIRST: specifies that NULL values should be returned before
non-NULL values.
NULLS LAST: specifies that NULL values should be returned after
non-NULL values.

5 weeks agoFix InsertStmt reAnalyze bug (#251)
kangkaisen [Mon, 5 Nov 2018 07:36:40 +0000 (15:36 +0800)] 
Fix InsertStmt reAnalyze bug (#251)

5 weeks agoFix typo in READEME.md (#278)
lide [Mon, 5 Nov 2018 06:12:11 +0000 (14:12 +0800)] 
Fix typo in READEME.md (#278)

5 weeks agoFix BE can't be started by start_be.sh (#281)
chenhao7253886 [Mon, 5 Nov 2018 03:21:51 +0000 (11:21 +0800)] 
Fix BE can't be started by start_be.sh (#281)

5 weeks agoRemove my aes and fix palo ns to doris (#277)
kangpinghuang [Fri, 2 Nov 2018 09:05:48 +0000 (17:05 +0800)] 
Remove my aes and fix palo ns to doris (#277)

5 weeks agoFix a bug that user can not kill it own connection (#276)
morningman [Fri, 2 Nov 2018 08:36:59 +0000 (16:36 +0800)] 
Fix a bug that user can not kill it own connection (#276)

5 weeks agoChange SQL built-in function's symbol (#274)
chenhao7253886 [Fri, 2 Nov 2018 08:24:21 +0000 (16:24 +0800)] 
Change SQL built-in function's symbol (#274)

5 weeks agoDelete useless debug log (#250)
kangkaisen [Fri, 2 Nov 2018 08:06:01 +0000 (16:06 +0800)] 
Delete useless debug log (#250)

5 weeks agoRewrite aes encryption (#264)
kangpinghuang [Fri, 2 Nov 2018 07:26:31 +0000 (15:26 +0800)] 
Rewrite aes encryption (#264)

Resolve #257

5 weeks agoUpdate README.md and build.sh (#273)
lide [Fri, 2 Nov 2018 07:23:44 +0000 (15:23 +0800)] 
Update README.md and build.sh (#273)

change some 'palo' to 'doris'

5 weeks agoFix UnionStmt toSql bug (#249)
kangkaisen [Fri, 2 Nov 2018 06:50:09 +0000 (14:50 +0800)] 
Fix UnionStmt toSql bug (#249)

5 weeks agoFix unit test failed in BE (#269)
chenhao7253886 [Fri, 2 Nov 2018 05:03:15 +0000 (13:03 +0800)] 
Fix unit test failed in BE (#269)

5 weeks agoChange variable and namespace name in BE (#268)
chenhao7253886 [Fri, 2 Nov 2018 02:22:32 +0000 (10:22 +0800)] 
Change variable and namespace name in BE (#268)

Change 'palo' to 'doris'

5 weeks agoFailed to register equal conjuncts which refer more than three tuples (#266)
chenhao7253886 [Thu, 1 Nov 2018 12:34:48 +0000 (20:34 +0800)] 
Failed to register equal conjuncts which refer more than three tuples (#266)

Change-Id: I7eaf28ee6db35671971108f3edefe908d46ae87f

5 weeks agoMake class name compatible with the old name (#267)
morningman [Thu, 1 Nov 2018 11:23:10 +0000 (19:23 +0800)] 
Make class name compatible with the old name (#267)

We wrote the old name 'com.baidu.palo.xxx' in bdbje as meta journal.
So we need to make it compatible with 'org.apache.doris.xxx'.

5 weeks agoChange the lock type of Catalog lock (#265)
morningman [Thu, 1 Nov 2018 08:33:09 +0000 (16:33 +0800)] 
Change the lock type of Catalog lock (#265)

* Change the lock type of Catalog lock

Implement a QueryableReentrantLock to see which thread held the lock.

* Key in empty version has no min/max value

It should be ignored to reconstruct min/max statistics upon restart.

5 weeks agoTidy up the docs and gensrc directory (#263)
morningman [Thu, 1 Nov 2018 02:37:30 +0000 (10:37 +0800)] 
Tidy up the docs and gensrc directory (#263)

1. Remove all design docs. They will be pushed again after modification.
2. Add streaming load and privilege help docs.
3. Rename palo*.py to doris*.py in gensrc/script/.

5 weeks agoChange license to Apache License 2.0 (#262)
morningman [Thu, 1 Nov 2018 01:06:01 +0000 (09:06 +0800)] 
Change license to Apache License 2.0 (#262)