网上有很多关于pos机汇总异常,Spark Graphx 图计算 图构造器汇总的知识，也有很多人为大家解答关于pos机汇总异常的问题，今天pos机之家(www.poszjia.com)为大家整理了关于这方面的知识，让我们一起来看下吧!
本文目录一览：
1、pos机汇总异常
pos机汇总异常
spark grahx构造图的方式汇总如下：
1.最原始的方法构造graph使用Graph的伴生对象的apply方法。允许从顶点和边的RDD上创建一个图。
import org.apache.log4j.{Level, Logger}import org.apache.spark.graphx.{Graph, Edge}import org.apache.spark.{SparkContext, SparkConf}/*** Created by lichangyue on 2016/9/13.*/object FirstGraph1 {def main(args: Array[String]) { Logger.getLogger("org.apache.spark").setLevel(Level.ERROR); Logger.getLogger("org.eclipse.jetty.server").setLevel(Level.ERROR);val conf = new SparkConf().setAppName("FirstGrahp").setMaster("local")val sc = new SparkContext(conf)val users = sc.parallelize(Array((3L,("rxin","student")),(7L,("jgonzal","postdoc")), (5L,("franklin","prof")),(2L,("istoica","prof"))))val relationships = sc.parallelize(Array(Edge(3L,7L,"collab"),Edge(5L,3L,"advisor"),Edge(2L,5L,"collegaue"),Edge(5L,7L,"pi")))val defaultUser = ("John Doe","Missing")
//创建图val graph =Graph(users,relationships,defaultUser)//模式匹配 println( "count :" +graph.vertices.filter{ case(id,(name,pos)) => pos =="postdoc"}.count)graph.edges.filter(e => e.srcId > e.dstId ).count() }}2.从文件中读取读取数据构建graph
从文件中读取读取数据构建graph的接口，GraphLoader中的edgeListFile方法。
GraphLoader.edgeListFile 从文件中加载一张图，文件内容如下：
2 1
4 1
1 2
会自动根据边创建顶点，所有的边和顶点的属性默认为1。
object GraphFromFile2 {def main(args: Array[String]) { Logger.getLogger("org.apache.spark").setLevel(Level.ERROR); Logger.getLogger("org.eclipse.jetty.server").setLevel(Level.ERROR);// val conf = new SparkConf().setAppName("graph2").setMaster("spark://10.58.22.219:7077")val conf = new SparkConf().setAppName("graph2").setMaster("local[4]")val sc = new SparkContext(conf)// val graph = GraphLoader.edgeListFile(sc,"hdfs://S7SA053:8020/stat/web-Google.txt") //指定分区数量val graph = GraphLoader.edgeListFile(sc,"hdfs://S7SA053:8020/stat/web-Google.txt",numEdgepartitions=4)//查看顶点数量val vcount = graph.vertices.count()println("vcount:" +vcount)//查看边的数量val ecount = graph.edges.count()println("ecount:" + ecount) }}3. 使用边创建图
Graph.fromEdges(ClassTag[VD],ClassTag[ED]):Graph[VD,ED]) 允许仅仅从一个边RDD上创建一个图，它自动地创建边提及的顶点，并分配这些顶点默认的值。
def fromEdges[VD: ClassTag, ED: ClassTag]( edges: RDD[Edge[ED]], defaultValue: VD, edgeStorageLevel: StorageLevel = StorageLevel.MEMORY_ONLY, vertexStorageLevel: StorageLevel = StorageLevel.MEMORY_ONLY): Graph[VD, ED]
val edge=List(//边的信息(111,122),(111,133),(122,133),(133,144),(133,155),(133,116), (144,155),(155,116),(177,188),(177,199),(188,199))//构建边的rddval edgeRdd = sc.parallelize(edge).map(x =>{Edge(x._1.toLong , x._2.toLong, None)})//通过边构建图 ， 0是顶点默认属性val graph = Graph.fromEdges(edgeRdd,0)4.从源点和目的点的元组构建
Graph.fromEdgeTuples],VD,Option[PartitionStrategy])(ClassTag[VD]):Graph[VD,Int]) 允许仅仅从一个边元组组成的RDD上创建一个图。分配给边的值为1。它自动地创建边提及的顶点，并分配这些顶点默认的值。
def fromEdgeTuples[VD: ClassTag]( rawEdges: RDD[(VertexId, VertexId)], defaultValue: VD, uniqueEdges: Option[PartitionStrategy] = None, edgeStorageLevel: StorageLevel = StorageLevel.MEMORY_ONLY, vertexStorageLevel: StorageLevel = StorageLevel.MEMORY_ONLY): Graph[VD, Int]5.生成随机图
GraphGenerators.logNormalGraph随机图生成方法源码：默认出度为4，标准偏差为1.3，并行生成numVertices，partition默认为sc的默认partition。
Logger.getLogger("org.apache.spark").setLevel(Level.ERROR);Logger.getLogger("org.eclipse.jetty.server").setLevel(Level.ERROR);val conf = new SparkConf()val sc = new SparkContext("local","text",conf)val graph = GraphGenerators.logNormalGraph(sc,numVertices = 10) //生成10个顶点的图 .mapVertices((id,_) => id.toDouble)graph.vertices.collect.foreach(println(_))graph.edges.collect.foreach(println(_))
以上就是关于pos机汇总异常,Spark Graphx 图计算 图构造器汇总的知识，后面我们会继续为大家整理关于pos机汇总异常的知识，希望能够帮助到大家！
