面试题收集 - Java - 2017

Posted on 2017-08-08 Edited on 2025-07-25 In Tech Symbols count in article: 4k Reading time ≈ 4 mins.

我只是收集一下题目，检查一下自己。

京东平台产品研发部 - Java

背景：一年工作经验，做电子政务

一个数组 i[] = [-1000 ~ 1000] 中的任意一些数字，求乘积最大的三个数。
static都能用在哪里？
1M 等于多少个1
面向对象的特性
写出两种单例模式
HashMap有哪些方法
HashMap的底层实现
List和Set的区别
重载和重写的区别，多态如何体现
判断字符串是否含有此Url：www.jd.com

Long l1 = new Long(1024L);
Long l2 = new Long(1024L);
System.out.println(l1 == l2);//false
Integer i1 = 100;
Integer i2 = 100;
System.out.println(i1 == i2);//true
i1 = 100;
i2 = 100;
System.out.println(i1 == i2);//true

Spring自定义注解、拦截器相关
为什么使用注解、有什么好处
Spring中运用了哪些设计模式
Sping和SpringBoot各有什么优缺点
Spring的事物隔离机制
说说AOP、DI、IOC
redis等用过没
缓存和memorycache
http://www.jd.com/...username=… 为了防止sql注入，怎么写正则。
mybatis如何防止sql注入
mybatis使用变量时怎么表示
使用js遍历json数据
使用Map遍历json数据
elasticsearch相关
你曾经在社区接触过的技术有哪些
项目相关：做了哪些部分、职责、用到哪些技术、设计了哪些
介绍一下RPC调用

谷歌淘来的一个总结

面试题总结 —— JAVA高级工程师

部分答案

有道云笔记支持的markdown语法 - hexo试一试

Posted on 2017-08-07 Edited on 2025-07-25 Symbols count in article: 491 Reading time ≈ 1 mins.

markdown语法

分割线

列表1
列表2
列表3

//插入代码
public void main(String[] args){
    System.out.println("Hello World!");
}

一级标题

二级标题

三级标题

四级标题

五级标题

六级标题

~~删除线~~

引用，只能写在一行

未勾选
勾选的

梦殇国际

markdown源码

Spark第一个Java程序 - 字词统计 - Spark Java idea

Posted on 2017-08-02 Edited on 2025-07-25 In Tech Symbols count in article: 23k Reading time ≈ 21 mins.

在本地利用idea，Java开发spark程序

原来并不用安装spark什么的这些东西

这样就不会那么繁琐，门槛也低了点

具体过程如下

一、创建工程

idea -> new Project -> maven -> create from archetype -> maven-archetype-quickstart
pom.xml添加依赖

<dependency>
  <groupId>org.apache.spark</groupId>
  <artifactId>spark-core_2.11</artifactId>
  <version>2.0.1</version>
</dependency>

编写Java代码

package com.hisen.spark;

import org.apache.spark.SparkConf;
import org.apache.spark.api.Java.JavaRDD;
import org.apache.spark.api.Java.JavaSparkContext;
import org.apache.spark.api.Java.function.Function;

/**
 * 第一个spark程序
 * Created by hisenyuan on 2017/8/2 at 16:36.
 */
public class SimpleApp {

  public static void main(String[] args) {
    // Should be some file on your system
    String logFile = "D:\\logs\\boss_debug.log";
    //设置本地运行，设置名称（在spark web ui上显示）
    SparkConf sparkConf = new SparkConf().setMaster("local").setAppName("Simple Application");
    JavaSparkContext JavaSparkContext = new JavaSparkContext(sparkConf);
    JavaRDD<String> logData = JavaSparkContext.textFile(logFile).cache();

    //统计包含a的次数
    long countA = logData.filter(new Function<String, Boolean>() {
      public Boolean call(String s) throws Exception {
        return s.contains("a");
      }
    }).count();

    //统计包含b的次数
    long countB = logData.filter(new Function<String, Boolean>() {
      public Boolean call(String s) throws Exception {
        return s.contains("b");
      }
    }).count();
    System.out.printf("Lines with a: %d, lines with b: %d\n", countA, countB);
    //Lines with a: 11146, lines with b: 10760
  }
}

运行main方法：结果：Lines with a: 11146, lines with b: 10760
日志如下：

C:\hisenwork\soft\jdk8\bin\Java -Dspark.master=local "-Javaagent:C:\hisenwork\IntelliJ IDEA 2017.1.1\lib\idea_rt.jar=59366:C:\hisenwork\IntelliJ IDEA 2017.1.1\bin" -Dfile.encoding=UTF-8 -classpath C:\hisenwork\soft\jdk8\jre\lib\charsets.jar;C:\hisenwork\soft\jdk8\jre\lib\deploy.jar;C:\hisenwork\soft\jdk8\jre\lib\ext\access-bridge-64.jar;C:\hisenwork\soft\jdk8\jre\lib\ext\cldrdata.jar;C:\hisenwork\soft\jdk8\jre\lib\ext\dnsns.jar;C:\hisenwork\soft\jdk8\jre\lib\ext\jaccess.jar;C:\hisenwork\soft\jdk8\jre\lib\ext\jfxrt.jar;C:\hisenwork\soft\jdk8\jre\lib\ext\localedata.jar;C:\hisenwork\soft\jdk8\jre\lib\ext\nashorn.jar;C:\hisenwork\soft\jdk8\jre\lib\ext\sunec.jar;C:\hisenwork\soft\jdk8\jre\lib\ext\sunjce_provider.jar;C:\hisenwork\soft\jdk8\jre\lib\ext\sunmscapi.jar;C:\hisenwork\soft\jdk8\jre\lib\ext\sunpkcs11.jar;C:\hisenwork\soft\jdk8\jre\lib\ext\zipfs.jar;C:\hisenwork\soft\jdk8\jre\lib\Javaws.jar;C:\hisenwork\soft\jdk8\jre\lib\jce.jar;C:\hisenwork\soft\jdk8\jre\lib\jfr.jar;C:\hisenwork\soft\jdk8\jre\lib\jfxswt.jar;C:\hisenwork\soft\jdk8\jre\lib\jsse.jar;C:\hisenwork\soft\jdk8\jre\lib\management-agent.jar;C:\hisenwork\soft\jdk8\jre\lib\plugin.jar;C:\hisenwork\soft\jdk8\jre\lib\resources.jar;C:\hisenwork\soft\jdk8\jre\lib\rt.jar;C:\hisenwork\code\rmitec\SparkTest\target\classes;C:\hisenwork\soft\maven\org\apache\spark\spark-core_2.11\2.0.1\spark-core_2.11-2.0.1.jar;C:\hisenwork\soft\maven\org\apache\avro\avro-mapred\1.7.7\avro-mapred-1.7.7-hadoop2.jar;C:\hisenwork\soft\maven\org\apache\avro\avro-ipc\1.7.7\avro-ipc-1.7.7.jar;C:\hisenwork\soft\maven\org\apache\avro\avro\1.7.7\avro-1.7.7.jar;C:\hisenwork\soft\maven\org\apache\avro\avro-ipc\1.7.7\avro-ipc-1.7.7-tests.jar;C:\hisenwork\soft\maven\org\codehaus\jackson\jackson-core-asl\1.9.13\jackson-core-asl-1.9.13.jar;C:\hisenwork\soft\maven\org\codehaus\jackson\jackson-mapper-asl\1.9.13\jackson-mapper-asl-1.9.13.jar;C:\hisenwork\soft\maven\com\twitter\chill_2.11\0.8.0\chill_2.11-0.8.0.jar;C:\hisenwork\soft\maven\com\esotericsoftware\kryo-shaded\3.0.3\kryo-shaded-3.0.3.jar;C:\hisenwork\soft\maven\com\esotericsoftware\minlog\1.3.0\minlog-1.3.0.jar;C:\hisenwork\soft\maven\org\objenesis\objenesis\2.1\objenesis-2.1.jar;C:\hisenwork\soft\maven\com\twitter\chill-Java\0.8.0\chill-Java-0.8.0.jar;C:\hisenwork\soft\maven\org\apache\xbean\xbean-asm5-shaded\4.4\xbean-asm5-shaded-4.4.jar;C:\hisenwork\soft\maven\org\apache\hadoop\hadoop-client\2.2.0\hadoop-client-2.2.0.jar;C:\hisenwork\soft\maven\org\apache\hadoop\hadoop-common\2.2.0\hadoop-common-2.2.0.jar;C:\hisenwork\soft\maven\commons-cli\commons-cli\1.2\commons-cli-1.2.jar;C:\hisenwork\soft\maven\org\apache\commons\commons-math\2.1\commons-math-2.1.jar;C:\hisenwork\soft\maven\xmlenc\xmlenc\0.52\xmlenc-0.52.jar;C:\hisenwork\soft\maven\commons-io\commons-io\2.1\commons-io-2.1.jar;C:\hisenwork\soft\maven\commons-lang\commons-lang\2.5\commons-lang-2.5.jar;C:\hisenwork\soft\maven\commons-configuration\commons-configuration\1.6\commons-configuration-1.6.jar;C:\hisenwork\soft\maven\commons-collections\commons-collections\3.2.1\commons-collections-3.2.1.jar;C:\hisenwork\soft\maven\commons-digester\commons-digester\1.8\commons-digester-1.8.jar;C:\hisenwork\soft\maven\commons-beanutils\commons-beanutils\1.7.0\commons-beanutils-1.7.0.jar;C:\hisenwork\soft\maven\commons-beanutils\commons-beanutils-core\1.8.0\commons-beanutils-core-1.8.0.jar;C:\hisenwork\soft\maven\com\google\protobuf\protobuf-Java\2.5.0\protobuf-Java-2.5.0.jar;C:\hisenwork\soft\maven\org\apache\hadoop\hadoop-auth\2.2.0\hadoop-auth-2.2.0.jar;C:\hisenwork\soft\maven\org\apache\commons\commons-compress\1.4.1\commons-compress-1.4.1.jar;C:\hisenwork\soft\maven\org\tukaani\xz\1.0\xz-1.0.jar;C:\hisenwork\soft\maven\org\apache\hadoop\hadoop-hdfs\2.2.0\hadoop-hdfs-2.2.0.jar;C:\hisenwork\soft\maven\org\mortbay\jetty\jetty-util\6.1.26\jetty-util-6.1.26.jar;C:\hisenwork\soft\maven\org\apache\hadoop\hadoop-mapreduce-client-app\2.2.0\hadoop-mapreduce-client-app-2.2.0.jar;C:\hisenwork\soft\maven\org\apache\hadoop\hadoop-mapreduce-client-common\2.2.0\hadoop-mapreduce-client-common-2.2.0.jar;C:\hisenwork\soft\maven\org\apache\hadoop\hadoop-yarn-client\2.2.0\hadoop-yarn-client-2.2.0.jar;C:\hisenwork\soft\maven\com\google\inject\guice\3.0\guice-3.0.jar;C:\hisenwork\soft\maven\Javax\inject\Javax.inject\1\Javax.inject-1.jar;C:\hisenwork\soft\maven\aopalliance\aopalliance\1.0\aopalliance-1.0.jar;C:\hisenwork\soft\maven\org\apache\hadoop\hadoop-yarn-server-common\2.2.0\hadoop-yarn-server-common-2.2.0.jar;C:\hisenwork\soft\maven\org\apache\hadoop\hadoop-mapreduce-client-shuffle\2.2.0\hadoop-mapreduce-client-shuffle-2.2.0.jar;C:\hisenwork\soft\maven\org\apache\hadoop\hadoop-yarn-api\2.2.0\hadoop-yarn-api-2.2.0.jar;C:\hisenwork\soft\maven\org\apache\hadoop\hadoop-mapreduce-client-core\2.2.0\hadoop-mapreduce-client-core-2.2.0.jar;C:\hisenwork\soft\maven\org\apache\hadoop\hadoop-yarn-common\2.2.0\hadoop-yarn-common-2.2.0.jar;C:\hisenwork\soft\maven\org\apache\hadoop\hadoop-mapreduce-client-jobclient\2.2.0\hadoop-mapreduce-client-jobclient-2.2.0.jar;C:\hisenwork\soft\maven\org\apache\hadoop\hadoop-annotations\2.2.0\hadoop-annotations-2.2.0.jar;C:\hisenwork\soft\maven\org\apache\spark\spark-launcher_2.11\2.0.1\spark-launcher_2.11-2.0.1.jar;C:\hisenwork\soft\maven\org\apache\spark\spark-network-common_2.11\2.0.1\spark-network-common_2.11-2.0.1.jar;C:\hisenwork\soft\maven\org\fusesource\leveldbjni\leveldbjni-all\1.8\leveldbjni-all-1.8.jar;C:\hisenwork\soft\maven\com\fasterxml\jackson\core\jackson-annotations\2.6.5\jackson-annotations-2.6.5.jar;C:\hisenwork\soft\maven\org\apache\spark\spark-network-shuffle_2.11\2.0.1\spark-network-shuffle_2.11-2.0.1.jar;C:\hisenwork\soft\maven\org\apache\spark\spark-unsafe_2.11\2.0.1\spark-unsafe_2.11-2.0.1.jar;C:\hisenwork\soft\maven\net\Java\dev\jets3t\jets3t\0.7.1\jets3t-0.7.1.jar;C:\hisenwork\soft\maven\commons-codec\commons-codec\1.3\commons-codec-1.3.jar;C:\hisenwork\soft\maven\commons-httpclient\commons-httpclient\3.1\commons-httpclient-3.1.jar;C:\hisenwork\soft\maven\org\apache\curator\curator-recipes\2.4.0\curator-recipes-2.4.0.jar;C:\hisenwork\soft\maven\org\apache\curator\curator-framework\2.4.0\curator-framework-2.4.0.jar;C:\hisenwork\soft\maven\org\apache\curator\curator-client\2.4.0\curator-client-2.4.0.jar;C:\hisenwork\soft\maven\org\apache\zookeeper\zookeeper\3.4.5\zookeeper-3.4.5.jar;C:\hisenwork\soft\maven\com\google\guava\guava\14.0.1\guava-14.0.1.jar;C:\hisenwork\soft\maven\Javax\servlet\Javax.servlet-api\3.1.0\Javax.servlet-api-3.1.0.jar;C:\hisenwork\soft\maven\org\apache\commons\commons-lang3\3.3.2\commons-lang3-3.3.2.jar;C:\hisenwork\soft\maven\org\apache\commons\commons-math3\3.4.1\commons-math3-3.4.1.jar;C:\hisenwork\soft\maven\com\google\code\findbugs\jsr305\1.3.9\jsr305-1.3.9.jar;C:\hisenwork\soft\maven\org\slf4j\slf4j-api\1.7.16\slf4j-api-1.7.16.jar;C:\hisenwork\soft\maven\org\slf4j\jul-to-slf4j\1.7.16\jul-to-slf4j-1.7.16.jar;C:\hisenwork\soft\maven\org\slf4j\jcl-over-slf4j\1.7.16\jcl-over-slf4j-1.7.16.jar;C:\hisenwork\soft\maven\log4j\log4j\1.2.17\log4j-1.2.17.jar;C:\hisenwork\soft\maven\org\slf4j\slf4j-log4j12\1.7.16\slf4j-log4j12-1.7.16.jar;C:\hisenwork\soft\maven\com\ning\compress-lzf\1.0.3\compress-lzf-1.0.3.jar;C:\hisenwork\soft\maven\org\xerial\snappy\snappy-Java\1.1.2.6\snappy-Java-1.1.2.6.jar;C:\hisenwork\soft\maven\net\jpountz\lz4\lz4\1.3.0\lz4-1.3.0.jar;C:\hisenwork\soft\maven\org\roaringbitmap\RoaringBitmap\0.5.11\RoaringBitmap-0.5.11.jar;C:\hisenwork\soft\maven\commons-net\commons-net\2.2\commons-net-2.2.jar;C:\hisenwork\soft\maven\org\scala-lang\scala-library\2.11.8\scala-library-2.11.8.jar;C:\hisenwork\soft\maven\org\json4s\json4s-jackson_2.11\3.2.11\json4s-jackson_2.11-3.2.11.jar;C:\hisenwork\soft\maven\org\json4s\json4s-core_2.11\3.2.11\json4s-core_2.11-3.2.11.jar;C:\hisenwork\soft\maven\org\json4s\json4s-ast_2.11\3.2.11\json4s-ast_2.11-3.2.11.jar;C:\hisenwork\soft\maven\com\thoughtworks\paranamer\paranamer\2.6\paranamer-2.6.jar;C:\hisenwork\soft\maven\org\scala-lang\scalap\2.11.0\scalap-2.11.0.jar;C:\hisenwork\soft\maven\org\scala-lang\scala-compiler\2.11.0\scala-compiler-2.11.0.jar;C:\hisenwork\soft\maven\org\scala-lang\modules\scala-parser-combinators_2.11\1.0.1\scala-parser-combinators_2.11-1.0.1.jar;C:\hisenwork\soft\maven\org\glassfish\jersey\core\jersey-client\2.22.2\jersey-client-2.22.2.jar;C:\hisenwork\soft\maven\Javax\ws\rs\Javax.ws.rs-api\2.0.1\Javax.ws.rs-api-2.0.1.jar;C:\hisenwork\soft\maven\org\glassfish\hk2\hk2-api\2.4.0-b34\hk2-api-2.4.0-b34.jar;C:\hisenwork\soft\maven\org\glassfish\hk2\hk2-utils\2.4.0-b34\hk2-utils-2.4.0-b34.jar;C:\hisenwork\soft\maven\org\glassfish\hk2\external\aopalliance-repackaged\2.4.0-b34\aopalliance-repackaged-2.4.0-b34.jar;C:\hisenwork\soft\maven\org\glassfish\hk2\external\Javax.inject\2.4.0-b34\Javax.inject-2.4.0-b34.jar;C:\hisenwork\soft\maven\org\glassfish\hk2\hk2-locator\2.4.0-b34\hk2-locator-2.4.0-b34.jar;C:\hisenwork\soft\maven\org\Javassist\Javassist\3.18.1-GA\Javassist-3.18.1-GA.jar;C:\hisenwork\soft\maven\org\glassfish\jersey\core\jersey-common\2.22.2\jersey-common-2.22.2.jar;C:\hisenwork\soft\maven\Javax\annotation\Javax.annotation-api\1.2\Javax.annotation-api-1.2.jar;C:\hisenwork\soft\maven\org\glassfish\jersey\bundles\repackaged\jersey-guava\2.22.2\jersey-guava-2.22.2.jar;C:\hisenwork\soft\maven\org\glassfish\hk2\osgi-resource-locator\1.0.1\osgi-resource-locator-1.0.1.jar;C:\hisenwork\soft\maven\org\glassfish\jersey\core\jersey-server\2.22.2\jersey-server-2.22.2.jar;C:\hisenwork\soft\maven\org\glassfish\jersey\media\jersey-media-jaxb\2.22.2\jersey-media-jaxb-2.22.2.jar;C:\hisenwork\soft\maven\Javax\validation\validation-api\1.1.0.Final\validation-api-1.1.0.Final.jar;C:\hisenwork\soft\maven\org\glassfish\jersey\containers\jersey-container-servlet\2.22.2\jersey-container-servlet-2.22.2.jar;C:\hisenwork\soft\maven\org\glassfish\jersey\containers\jersey-container-servlet-core\2.22.2\jersey-container-servlet-core-2.22.2.jar;C:\hisenwork\soft\maven\org\apache\mesos\mesos\0.21.1\mesos-0.21.1-shaded-protobuf.jar;C:\hisenwork\soft\maven\io\netty\netty-all\4.0.29.Final\netty-all-4.0.29.Final.jar;C:\hisenwork\soft\maven\io\netty\netty\3.8.0.Final\netty-3.8.0.Final.jar;C:\hisenwork\soft\maven\com\clearspring\analytics\stream\2.7.0\stream-2.7.0.jar;C:\hisenwork\soft\maven\io\dropwizard\metrics\metrics-core\3.1.2\metrics-core-3.1.2.jar;C:\hisenwork\soft\maven\io\dropwizard\metrics\metrics-jvm\3.1.2\metrics-jvm-3.1.2.jar;C:\hisenwork\soft\maven\io\dropwizard\metrics\metrics-json\3.1.2\metrics-json-3.1.2.jar;C:\hisenwork\soft\maven\io\dropwizard\metrics\metrics-graphite\3.1.2\metrics-graphite-3.1.2.jar;C:\hisenwork\soft\maven\com\fasterxml\jackson\core\jackson-databind\2.6.5\jackson-databind-2.6.5.jar;C:\hisenwork\soft\maven\com\fasterxml\jackson\core\jackson-core\2.6.5\jackson-core-2.6.5.jar;C:\hisenwork\soft\maven\com\fasterxml\jackson\module\jackson-module-scala_2.11\2.6.5\jackson-module-scala_2.11-2.6.5.jar;C:\hisenwork\soft\maven\org\scala-lang\scala-reflect\2.11.7\scala-reflect-2.11.7.jar;C:\hisenwork\soft\maven\com\fasterxml\jackson\module\jackson-module-paranamer\2.6.5\jackson-module-paranamer-2.6.5.jar;C:\hisenwork\soft\maven\org\apache\ivy\ivy\2.4.0\ivy-2.4.0.jar;C:\hisenwork\soft\maven\oro\oro\2.0.8\oro-2.0.8.jar;C:\hisenwork\soft\maven\net\razorvine\pyrolite\4.9\pyrolite-4.9.jar;C:\hisenwork\soft\maven\net\sf\py4j\py4j\0.10.3\py4j-0.10.3.jar;C:\hisenwork\soft\maven\org\apache\spark\spark-tags_2.11\2.0.1\spark-tags_2.11-2.0.1.jar;C:\hisenwork\soft\maven\org\scalatest\scalatest_2.11\2.2.6\scalatest_2.11-2.2.6.jar;C:\hisenwork\soft\maven\org\scala-lang\modules\scala-xml_2.11\1.0.2\scala-xml_2.11-1.0.2.jar;C:\hisenwork\soft\maven\org\spark-project\spark\unused\1.0.0\unused-1.0.0.jar com.hisen.spark.SimpleApp
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
17/08/02 17:00:09 INFO SparkContext: Running Spark version 2.0.1
17/08/02 17:00:10 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-Java classes where applicable
17/08/02 17:00:10 INFO SecurityManager: Changing view acls to: Administrator
17/08/02 17:00:10 INFO SecurityManager: Changing modify acls to: Administrator
17/08/02 17:00:10 INFO SecurityManager: Changing view acls groups to:
17/08/02 17:00:10 INFO SecurityManager: Changing modify acls groups to:
17/08/02 17:00:10 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(Administrator); groups with view permissions: Set(); users  with modify permissions: Set(Administrator); groups with modify permissions: Set()
17/08/02 17:00:11 INFO Utils: Successfully started service 'sparkDriver' on port 59388.
17/08/02 17:00:11 INFO SparkEnv: Registering MapOutputTracker
17/08/02 17:00:11 INFO SparkEnv: Registering BlockManagerMaster
17/08/02 17:00:11 INFO DiskBlockManager: Created local directory at C:\Users\Administrator\AppData\Local\Temp\blockmgr-630d8294-e5cb-428d-8189-4c3313e46fa3
17/08/02 17:00:12 INFO MemoryStore: MemoryStore started with capacity 894.3 MB
17/08/02 17:00:12 INFO SparkEnv: Registering OutputCommitCoordinator
17/08/02 17:00:12 INFO Utils: Successfully started service 'SparkUI' on port 4040.
17/08/02 17:00:12 INFO SparkUI: Bound SparkUI to 0.0.0.0, and started at http://169.254.90.236:4040
17/08/02 17:00:12 INFO Executor: Starting executor ID driver on host localhost
17/08/02 17:00:12 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 59397.
17/08/02 17:00:12 INFO NettyBlockTransferService: Server created on 169.254.90.236:59397
17/08/02 17:00:12 INFO BlockManagerMaster: Registering BlockManager BlockManagerId(driver, 169.254.90.236, 59397)
17/08/02 17:00:12 INFO BlockManagerMasterEndpoint: Registering block manager 169.254.90.236:59397 with 894.3 MB RAM, BlockManagerId(driver, 169.254.90.236, 59397)
17/08/02 17:00:12 INFO BlockManagerMaster: Registered BlockManager BlockManagerId(driver, 169.254.90.236, 59397)
17/08/02 17:00:14 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 127.1 KB, free 894.2 MB)
17/08/02 17:00:14 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 14.3 KB, free 894.2 MB)
17/08/02 17:00:14 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on 169.254.90.236:59397 (size: 14.3 KB, free: 894.3 MB)
17/08/02 17:00:14 INFO SparkContext: Created broadcast 0 from textFile at SimpleApp.Java:19
17/08/02 17:00:14 ERROR Shell: Failed to locate the winutils binary in the hadoop binary path
Java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
	at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.Java:278)
	at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.Java:300)
	at org.apache.hadoop.util.Shell.<clinit>(Shell.Java:293)
	at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.Java:76)
	at org.apache.hadoop.mapred.FileInputFormat.setInputPaths(FileInputFormat.Java:362)
	at org.apache.spark.SparkContext$$anonfun$hadoopFile$1$$anonfun$29.apply(SparkContext.scala:992)
	at org.apache.spark.SparkContext$$anonfun$hadoopFile$1$$anonfun$29.apply(SparkContext.scala:992)
	at org.apache.spark.rdd.HadoopRDD$$anonfun$getJobConf$6.apply(HadoopRDD.scala:176)
	at org.apache.spark.rdd.HadoopRDD$$anonfun$getJobConf$6.apply(HadoopRDD.scala:176)
	at scala.Option.map(Option.scala:146)
	at org.apache.spark.rdd.HadoopRDD.getJobConf(HadoopRDD.scala:176)
	at org.apache.spark.rdd.HadoopRDD.getPartitions(HadoopRDD.scala:195)
	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:248)
	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:246)
	at scala.Option.getOrElse(Option.scala:121)
	at org.apache.spark.rdd.RDD.partitions(RDD.scala:246)
	at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35)
	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:248)
	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:246)
	at scala.Option.getOrElse(Option.scala:121)
	at org.apache.spark.rdd.RDD.partitions(RDD.scala:246)
	at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35)
	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:248)
	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:246)
	at scala.Option.getOrElse(Option.scala:121)
	at org.apache.spark.rdd.RDD.partitions(RDD.scala:246)
	at org.apache.spark.SparkContext.runJob(SparkContext.scala:1930)
	at org.apache.spark.rdd.RDD.count(RDD.scala:1134)
	at org.apache.spark.api.Java.JavaRDDLike$class.count(JavaRDDLike.scala:454)
	at org.apache.spark.api.Java.AbstractJavaRDDLike.count(JavaRDDLike.scala:45)
	at com.hisen.spark.SimpleApp.main(SimpleApp.Java:26)
17/08/02 17:00:14 INFO FileInputFormat: Total input paths to process : 1
17/08/02 17:00:14 INFO SparkContext: Starting job: count at SimpleApp.Java:26
17/08/02 17:00:15 INFO DAGScheduler: Got job 0 (count at SimpleApp.Java:26) with 1 output partitions
17/08/02 17:00:15 INFO DAGScheduler: Final stage: ResultStage 0 (count at SimpleApp.Java:26)
17/08/02 17:00:15 INFO DAGScheduler: Parents of final stage: List()
17/08/02 17:00:15 INFO DAGScheduler: Missing parents: List()
17/08/02 17:00:15 INFO DAGScheduler: Submitting ResultStage 0 (MapPartitionsRDD[2] at filter at SimpleApp.Java:22), which has no missing parents
17/08/02 17:00:15 INFO MemoryStore: Block broadcast_1 stored as values in memory (estimated size 3.2 KB, free 894.2 MB)
17/08/02 17:00:15 INFO MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 1966.0 B, free 894.2 MB)
17/08/02 17:00:15 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on 169.254.90.236:59397 (size: 1966.0 B, free: 894.3 MB)
17/08/02 17:00:15 INFO SparkContext: Created broadcast 1 from broadcast at DAGScheduler.scala:1012
17/08/02 17:00:15 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 0 (MapPartitionsRDD[2] at filter at SimpleApp.Java:22)
17/08/02 17:00:15 INFO TaskSchedulerImpl: Adding task set 0.0 with 1 tasks
17/08/02 17:00:15 INFO TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, localhost, partition 0, PROCESS_LOCAL, 5324 bytes)
17/08/02 17:00:15 INFO Executor: Running task 0.0 in stage 0.0 (TID 0)
17/08/02 17:00:15 INFO HadoopRDD: Input split: file:/D:/logs/boss_debug.log:0+4059273
17/08/02 17:00:15 INFO deprecation: mapred.tip.id is deprecated. Instead, use mapreduce.task.id
17/08/02 17:00:15 INFO deprecation: mapred.task.id is deprecated. Instead, use mapreduce.task.attempt.id
17/08/02 17:00:15 INFO deprecation: mapred.task.is.map is deprecated. Instead, use mapreduce.task.ismap
17/08/02 17:00:15 INFO deprecation: mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
17/08/02 17:00:15 INFO deprecation: mapred.job.id is deprecated. Instead, use mapreduce.job.id
17/08/02 17:00:15 INFO MemoryStore: Block rdd_1_0 stored as values in memory (estimated size 5.7 MB, free 888.4 MB)
17/08/02 17:00:15 INFO BlockManagerInfo: Added rdd_1_0 in memory on 169.254.90.236:59397 (size: 5.7 MB, free: 888.5 MB)
17/08/02 17:00:15 INFO Executor: Finished task 0.0 in stage 0.0 (TID 0). 1842 bytes result sent to driver
17/08/02 17:00:16 INFO TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 691 ms on localhost (1/1)
17/08/02 17:00:16 INFO TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool
17/08/02 17:00:16 INFO DAGScheduler: ResultStage 0 (count at SimpleApp.Java:26) finished in 0.726 s
17/08/02 17:00:16 INFO DAGScheduler: Job 0 finished: count at SimpleApp.Java:26, took 1.078428 s
17/08/02 17:00:16 INFO SparkContext: Starting job: count at SimpleApp.Java:33
17/08/02 17:00:16 INFO DAGScheduler: Got job 1 (count at SimpleApp.Java:33) with 1 output partitions
17/08/02 17:00:16 INFO DAGScheduler: Final stage: ResultStage 1 (count at SimpleApp.Java:33)
17/08/02 17:00:16 INFO DAGScheduler: Parents of final stage: List()
17/08/02 17:00:16 INFO DAGScheduler: Missing parents: List()
17/08/02 17:00:16 INFO DAGScheduler: Submitting ResultStage 1 (MapPartitionsRDD[3] at filter at SimpleApp.Java:29), which has no missing parents
17/08/02 17:00:16 INFO MemoryStore: Block broadcast_2 stored as values in memory (estimated size 3.2 KB, free 888.4 MB)
17/08/02 17:00:16 INFO MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 1967.0 B, free 888.4 MB)
17/08/02 17:00:16 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on 169.254.90.236:59397 (size: 1967.0 B, free: 888.5 MB)
17/08/02 17:00:16 INFO SparkContext: Created broadcast 2 from broadcast at DAGScheduler.scala:1012
17/08/02 17:00:16 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 1 (MapPartitionsRDD[3] at filter at SimpleApp.Java:29)
17/08/02 17:00:16 INFO TaskSchedulerImpl: Adding task set 1.0 with 1 tasks
17/08/02 17:00:16 INFO TaskSetManager: Starting task 0.0 in stage 1.0 (TID 1, localhost, partition 0, PROCESS_LOCAL, 5324 bytes)
17/08/02 17:00:16 INFO Executor: Running task 0.0 in stage 1.0 (TID 1)
17/08/02 17:00:16 INFO BlockManager: Found block rdd_1_0 locally
17/08/02 17:00:16 INFO Executor: Finished task 0.0 in stage 1.0 (TID 1). 954 bytes result sent to driver
17/08/02 17:00:16 INFO DAGScheduler: ResultStage 1 (count at SimpleApp.Java:33) finished in 0.100 s
17/08/02 17:00:16 INFO DAGScheduler: Job 1 finished: count at SimpleApp.Java:33, took 0.139033 s
17/08/02 17:00:16 INFO TaskSetManager: Finished task 0.0 in stage 1.0 (TID 1) in 99 ms on localhost (1/1)
17/08/02 17:00:16 INFO TaskSchedulerImpl: Removed TaskSet 1.0, whose tasks have all completed, from pool

Lines with a: 11146, lines with b: 10760

17/08/02 17:00:16 INFO SparkContext: Invoking stop() from shutdown hook
17/08/02 17:00:16 INFO SparkUI: Stopped Spark web UI at http://169.254.90.236:4040
17/08/02 17:00:16 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
17/08/02 17:00:16 INFO MemoryStore: MemoryStore cleared
17/08/02 17:00:16 INFO BlockManager: BlockManager stopped
17/08/02 17:00:16 INFO BlockManagerMaster: BlockManagerMaster stopped
17/08/02 17:00:16 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
17/08/02 17:00:16 INFO SparkContext: Successfully stopped SparkContext
17/08/02 17:00:16 INFO ShutdownHookManager: Shutdown hook called
17/08/02 17:00:16 INFO ShutdownHookManager: Deleting directory C:\Users\Administrator\AppData\Local\Temp\spark-c40f2015-170f-4633-89dd-44dcd5bacfec

Process finished with exit code 0

Mybatis-Generator自动生成dao、model、mapper

Posted on 2017-08-02 Edited on 2025-07-25 In Tech Symbols count in article: 2.4k Reading time ≈ 2 mins.

一、配置插件

在resources文件夹下新建：generatorConfig.xml

内容如下：注意修改包名等信息

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE generatorConfiguration
  PUBLIC "-//mybatis.org//DTD MyBatis Generator Configuration 1.0//EN"
  "http://mybatis.org/dtd/mybatis-generator-config_1_0.dtd">
<!--
详细说明请看：http://blog.csdn.net/tiantangpw/article/details/51690534
-->
<generatorConfiguration>

  <context id="mysqlgenerator" targetRuntime="MyBatis3">
    <!--数据库配置-->
    <jdbcConnection connectionURL="jdbc:mysql://127.0.0.1:3306/booksystem"
      driverClass="com.mysql.jdbc.Driver"
      password="hisen"
      userId="root"/>
      
    <!--生成Model(实体类，与数据库字段对应的bean)类存放位置-->
    <JavaModelGenerator targetPackage="com.hisen.entity" targetProject="src/main/Java">
      <property name="enableSubPackages" value="true"/>
      <property name="trimStrings" value="true"/>
    </JavaModelGenerator>
    <!--生成映射(xxxmapper.xml)文件存放位置-->
    <sqlMapGenerator targetPackage="mapper" targetProject="src/main/resources">
      <property name="enableSubPackages" value="true"/>
    </sqlMapGenerator>
    <!--生成Dao类存放位置-->
    <JavaClientGenerator type="XMLMAPPER" targetPackage="com.hisen.dao"
      targetProject="src/main/Java">
      <property name="enableSubPackages" value="true"/>
    </JavaClientGenerator>
    
    <!--要生成的表-->
    <table tableName="appointment"/>
    <table tableName="user"/>
  </context>

</generatorConfiguration>

二、添加maven插件

mybatis - Invalid bound statement (not found)

Posted on 2017-08-02 Edited on 2025-07-25 In Tech Symbols count in article: 257 Reading time ≈ 1 mins.

运行报错

1	org.apache.ibatis.binding.BindingException: Invalid bound statement (not found): com.hisen.dao.UserMapper.insert

使用mybatis生成插件，产生的mapper，由于路径不对移动了一下mapper.Java文件

所以造成mapper.xml里面的namespace错误，无法映射

所以把namespace改为正确的即可

例如：

mapper.UserMapper

改为

com.hisen.dao.UserMapper

为什么要使用队列 - Java

Posted on 2017-08-01 Edited on 2025-07-25 In Tech Symbols count in article: 1k Reading time ≈ 1 mins.

一、Java中的队列：Queue接口

Queue接口与List、Set同一级别，都是继承了Collection接口。LinkedList实现了Queue接口。Queue接口窄化了对LinkedList的方法的访问权限（即在方法中的参数类型如果是Queue时，就完全只能访问Queue接口所定义的方法了，而不能直接访问 LinkedList的非Queue的方法），以使得只有恰当的方法才可以使用。BlockingQueue 继承了Queue接口。

队列是一种数据结构．它有两个基本操作：在队列尾部加入一个元素，和从队列头部移除一个元素（注意不要弄混队列的头部和尾部）就是说，队列以一种先进先出的方式管理数据，如果你试图向一个已经满了的阻塞队列中添加一个元素或者是从一个空的阻塞队列中移除一个元索，将导致线程阻塞．在多线程进行合作时，阻塞队列是很有用的工具。工作者线程可以定期地把中间结果存到阻塞队列中而其他工作者线程把中间结果取出并在将来修改它们。队列会自动平衡负载。如果第一个线程集运行得比第二个慢，则第二个线程集在等待结果时就会阻塞。如果第一个线程集运行得快，那么它将等待第二个线程集赶上来。下表显示了jdk1.5中的阻塞队列的操作：

排序方法	平均情况	最好情况
add	增加一个元素	如果队列已满，则抛出一个IllegalSlabEepeplian异常
remove	移除并返回队列头部的元素	如果队列为空，则抛出一个NoSuchElementException异常
element	返回队列头部的元素	如果队列为空，则抛出一个NoSuchElementException异常
offer	添加一个元素并返回true	如果队列已满，则返回false
poll	移除并返问队列头部的元素	如果队列为空，则返回null
peek	返回队列头部的元素	如果队列为空，则返回null
put	返回队列头部的元素	如果队列满，则阻塞
take	返回队列头部的元素	如果队列为空，则阻塞

二、消息队列

介绍：

消息队列中间件是分布式系统中重要的组件，主要解决应用耦合，异步消息，流量削锋等问题。

实现高性能，高可用，可伸缩和最终一致性架构。

是大型分布式系统不可缺少的中间件。

目前在生产环境，使用较多的消息队列有ActiveMQ，RabbitMQ，ZeroMQ，Kafka，MetaMQ，RocketMQ等。

场景：异步处理、应用解耦、流量削锋、日志处理

以上就是关于为什么要使用队列的大致说明

参考：

No MyBatis mapper was found in 'com.xxx.dao' package

Posted on 2017-08-01 Edited on 2025-07-25 In Tech Symbols count in article: 1.7k Reading time ≈ 2 mins.

出错

maven项目，启动tomcat的时候报错：

1	No MyBatis mapper was found in '[com.hisen.dao]' package

这是由于把mybatis的mapper配置文件放在了Java代码的目录下

目录结构

├── Java
│   └── com
│       └── hisen
│           ├── dao
│           │   └── BookDao.Java
│           │   └── BookMapper.xml
│           ├── entity
│           │   └── Book.Java
│           ├── service
│           │   ├── BookService.Java
│           │   └── impl
│           │       └── BookServiceImpl.Java
│           └── web
│               └── BookController.Java
├── resources
│   ├── jdbc.properties
│   ├── logback.xml
│   ├── mybatis-config.xml
│   └── spring
│       ├── spring-dao.xml
│       ├── spring-service.xml
│       └── spring-web.xml
└── webapp
    ├── index.jsp
    └── WEB-INF
        ├── jsp
        │   ├── detail.jsp
        │   └── list.jsp
        └── web.xml

出错原因

SpringMVC把jsp改为Thymeleaf模版引擎

Posted on 2017-08-01 Edited on 2025-07-25 In Tech Symbols count in article: 2.9k Reading time ≈ 3 mins.

Thymeleaf简介

前面的例子我们使用的视图技术主要是JSP。JSP的优点是它是Java EE容器的一部分，几乎所有Java EE服务器都支持JSP。缺点就是它在视图表现方面的功能很少，假如我们想迭代一个数组之类的，只能使用<% %>来包括Java语句进行。虽然有标准标签库（JSTL）的补足，但是使用仍然不太方便。另外JSP只能在Java EE容器中使用，如果我们希望渲染电子邮件之类的，JSP就无能为力了。

Java生态圈广泛，自然有很多视图框架，除了JSP之外，还有Freemarker、Velocity、Thymeleaf等很多框架。Thymeleaf的优点是它是基于HTML的，即使视图没有渲染成功，也是一个标准的HTML页面。因此它的可读性很不错，也可以作为设计原型来使用。而且它是完全独立于Java ee容器的，意味着我们可以在任何需要渲染HTML的地方使用Thymeleaf。

Thymeleaf也提供了spring的支持，我们可以非常方便的在Spring配置文件中声明Thymeleaf Beans，然后用它们渲染视图。

改造 - 由jsp到Thymeleaf

引入依赖

<!--thymeleaf模版 spring4.x-->
<dependency>
  <groupId>org.thymeleaf</groupId>
  <artifactId>thymeleaf-spring4</artifactId>
  <version>3.0.5.RELEASE</version>
</dependency>

配置ViewResolver(在spring的xml文件里)

Google Java工具类 - Guava - 常见用法

Posted on 2017-08-01 Edited on 2025-07-25 In Tech Symbols count in article: 4.1k Reading time ≈ 4 mins.

概述

工具类就是封装平常用的方法，不需要你重复造轮子，节省开发人员时间，提高工作效率。谷歌作为大公司，当然会从日常的工作中提取中很多高效率的方法出来。所以就诞生了Guava。

高效设计良好的API，被Google的开发者设计，实现和使用

遵循高效的Java语法实践

使代码更刻度，简洁，简单

节约时间，资源，提高生产力 Guava工程

包含了若干被Google的 Java项目广泛依赖的核心库，例如：

集合 [collections]
缓存 [caching]
原生类型支持 [primitives support]
并发库 [concurrency libraries]
通用注解 [common annotations]
字符串处理 [string processing]
I/O 等等。

部分用法如下：

MySQL update语句

Posted on 2017-08-01 Edited on 2025-07-25 In Tech Symbols count in article: 324 Reading time ≈ 1 mins.

刚在一个群里，有人有这么一个需求。

表A：id,name

表B：id,其他,name(新增字段)

A，B表通过id关联，要把A的name给对应的B的name

以前也没有写过这种update语句

update 表名 set 字段名=字段值 where 条件
如 update A set name='xiaoming' where id='';
如果是多表查询
update 表1 a inner join 表2 b on  ab表的关联 set a.字段=b.字段
如 update A a inner join B b on a.id=b.id set a.name=b.name
就是在table1表和table2表id相同时 把table2的name值赋给table1的name