Monthly Archives: May 2020

CSCI4430/ESTR4120 (Spring 2020) Assignment 3: Implementing NAT using NFQUEUE

Introduction

CSCI4430/ESTR4120 (Spring 2020) Assignment 3: Implementing NAT using NFQUEUE

Due on May 21, 2020 (Thur), 23:59:59

In this assignment, you will implement a NAT application using the software library NFQUEUE. The NAT application can forward UDP traffic.

1 Setup 1.1 Topology

Inbound –> packets ——–

Dept | network| ——–

| <– Outbound
| packets
| ———
| |VMA | <—–VMB/VMC | | (NAT) | UDP

| ———

Figure 1: Setup of the architecture.

Figure 1 shows the setup of this assignment. VM A has a network card configured with a public IP address (i.e., Apublic), and it will serve as the NAT gateway.

By properly configuring the route tables of VM B and VM C, VM A can relay any inbound/outbound traffic for VM B and VM C. Let Ain be the internal IP address of VM A. Then we execute the following command in both VM B and VM C to add the default gateway:

sudo route add default gw Ain
Note that you are NOT allowed to use any built-in NAT services, including the nat table in iptables.

Otherwise, you will get zero marks.

1.2 Assumptions and Restrictions

We make the following assumptions and restrictions.

  • You only need to relay UDP traffic. Other protocols (e.g., TCP and ICMP) can be ignored. We won’t generate any traffic aside UDP in our demo. To play safe, you can drop any unexpected packet.
  • All new flows are initiated from the internal network.
  • You only need to relay outbound traffic to one of the reachable workstations in the department. The reachable workstations include: linux5 – linux16.

1

IP=”10.3.1.54″
LAN=”10.0.54.0″
MASK=”24″
echo “1” > /proc/sys/net/ipv4/ip_forward iptables -t nat -F

# public interface
# private LAN network address (without subnet mask)
# subnet mask

iptables -t filter -F
iptables -t mangle -F
iptables -t filter -A FORWARD -j NFQUEUE –queue-num 0 -p udp -s ${LAN}/${MASK} \

! -d ${IP} –dport 10000:12000
iptables -t mangle -A PREROUTING -j NFQUEUE –queue-num 0 -p udp -d ${IP} \

–dport 10000:12000

1.3

• • •

Table 1: iptables configuration. Foreachreachableworkstation,youcanonlyhostaserveronport10000-12000(inclusive)using

UDP sockets.

The translated source port at VM A must also be in the range 10000 – 12000 (inclusive). We as- sume that no other processes are using the port 10000 – 12000 while the NAT program is running.

The NAT program maintains a translation table that keeps track of all NAT mappings of UDP. For each translation, the NAT program should always choose the smallest, available port number among all flows.

We will provide you the implementation of computing the checksums (for details, please refer to tutorials).

iptables

In VM A, you need to configure iptables to use NFQUEUE. Table 1 shows the rules to be used.
The first three iptables commands clear all the existing rules. The 4th and 5th iptables commands

redirect outbound and inbound UDP traffic, respectively.
The above script takes three inputs: (i) $IP is the IP address of public interface of VM A (i.e., Apublic

in Figure 1); (ii) $LAN is the network address of the private network (without the subnet mask); and (iii) $MASK is the subnet mask value. You must update the inputs for your own network.

2 UDP Translation

In UDP translation, the NAT program processes UDP packets as follows. 1. For each outbound packet,

  1. (a)  The NAT program searches if the source IP-port pair of the packet has already been stored in the translation table.
  2. (b)  If yes, the NAT program uses the previously assigned port number for the outbound packet.
  3. (c)  If not, then the NAT program creates a new entry in the translation table. The entry should contain: • the source IP-port pair;
    • the newly assigned port number (between 10000 and 12000) • the timestamp that the new entry is created.

2

(d) Finally, the NAT program translates the source IP address and the source port number of the packet, modifies the affected IP and UDP headers of the packet accordingly, and forwards the packet.

  1. For each inbound packet,
    1. (a)  The NAT program searches if the destination port of the inbound packet matches any one of the entries in the translation table.
    2. (b)  If yes, the NAT program translates its destination IP address and port number, modifies the IP and UDP headers of the packet accordingly, and sends the packet to the target VM.
    3. (c)  If not, the NAT program should drop the packet.
  2. Translation expiry requirement. For each entry, it should be valid for 10 seconds from the time since it is last accessed. In other words, if there is an inbound or outbound packet that matches the target entry, we restart the timer again. If no packet matches the target entry over the 10-second window, the entry should be removed. Here, we use a lazy approach to remove expired entries. Instead of maintaining a timer, we trigger the expiry process only when the NAT program receives a packet. When the NAT program receives a packet, it first checks and removes any entry that has not been accessed in the last 10 seconds, followed by performing translation on the packet. Some entries may have been idle for a while (e.g., 60 seconds), but it is fine to remove them later because they have no impact on the translation until a packet is processed. Note: Laziness is bad for study, but it’s a very useful technique in systems research for performance improvements without hurting the correctness.
  3. ICMPerrortranslation.IfanoutboundUDPpacketsendstoanclosedUDPport,thedestination may reply an ICMP port unreachable error. You don’t need to worry about this case.

To enable us to check the content of the NAT table, you should display all NAT mappings on the screen whenever there is an update in the NAT table (e.g., a new entry is added or an existing entry is deleted). Each displayed mapping should show the four fields: original source address, original source port, translated source address, and translated source port. You are free to define the display format. Also, make sure to remove other debugging messages from your submission for us to check the NAT table.

3 Traffic Shaping

We use token bucket to control the transmission rate.

  1. Token bucket. The token bucket holds a fixed number of logical tokens (bucket size). Tokens are generated and placed into the token bucket at a constant rate (fill rate), whose unit is the number of tokens generated per second (n/s). When the token bucket becomes full, subsequently generated tokens are discarded. So there are two parameters to configure a token bucket: bucket size and fill rate. The number of tokens in a bucket is initialized as the bucket size. When a packet arrives, transmit it if there is a token. Otherwise, wait until getting an available token. It consumes a token to transmit a packet.
  2. Threads. The NAT program leverages multi-threading, which has at least two threads:

(a) A thread for receiving packets. It receives packet from the queue we specified (i.e., queue 0 as shown in Table 1) and then handles it, which triggers the callback function. In the callback function, your program requires to check whether there is available buffer in user space. If yes, buffer the packet in user space; if not, drop the packet. Here we define the maximum number of packets in buffer as 10.

3

(b) A thread for processing packets. It conducts TCP translation for each buffered packet and sets verdict accordingly. Every transmission requires to get an available token from the token bucket.

4 Deliverables

Here are some possible testcases:
• Connections using “nc -u”, meaning that UDP traffic is used;
• Connections using “nc -u -p xxx”, meaning that the source port is set to “xxx”; • DNS lookup;
• Parallel sessions;
• Correctly displaying NAT mappings.

5 Submission

You are required to submit a set of C source codes that can be compiled into one executable file. The program should be successfully compiled without any warning message. We will provide the iptables script shown in Table 1 during grading. You must submit a Makefile to generate an executable file nat correctly. Your program should run as the following:

sudo ./nat <IP> <LAN> <MASK> <bucket size> <fill rate>.

Both the executable file and the script will be executed with the root privilege. Be sure that your program does not carry any “dangerous” command inside. Please refer to the submission guidelines on our course homepage.

4

程序代写代做 Hive javascript html data structure database Java CSC73010 – Programming Mobile Systems – Session 1 2020 Assignment 1

Posted on May 21, 2020 by mac

CSC73010 – Programming Mobile Systems – Session 1 2020 Assignment 1
Assignment 1: Creating client-end apps
Due Date: 20 April 2020 at 11.00 pm Submission Method: Blackboard Weight: 25% of Overall Grade Type: Individual

CSC73010 – Programming Mobile Systems – Session 1 2020 Assignment 1
Submission: You need to submit your source code as a ZIP archive for all parts of the assignment to assignment 1 submission link in the unit site (this is the preferred way). If the file size is too large to upload to submission link, please send the ZIP file via e-mail or via a method specified by your tutor. Note that marks will be deducted for poorly structured or uncommented code. All source code files submitted must include title comments that at least identify the author and the assignment part. The separate parts of the assignment are to be submitted in separate subdirectories (e.g. Part 1, Part 2) – submissions that ignore this instruction and leave all assignment files in one directory will be penalised.
Do not leave this assignment to the last minute – you can start on it while concepts are fresh in your mind. You may start Part 1 before you have studied all of the required materials. If you require an extension you must apply to your tutor before the due date to be considered. Unless an extension is approved there is a penalty (see Unit Information Guide).
Assignment Overview: This assignment requires you to develop an application using Typescript for Part 1, and Angular for Part 2, to demonstrate your knowledge of the Typescript language and Angular framework. Your apps should will be stand-alone apps without server contact for data. The app will not save data after the browser window is closed but you will need to maintain a JavaScript object so that changes made remain while the browser window is open.
Part 2 can be an expansion of Part 1 though you can start again from scratch if you wish. It is recommended that you save your Part 1 before you start modifying it for Part 2.
Part 1 – Basic TypeScript App (12 marks)
The aim is to develop a simple one-page TypeScript app to maintain a small database of ‘movie’ information while the browser window is open. You can initialise your app with hard coded data, or you can start with an empty data structure. Types are important in your code and you are expected to declare types for example, for variables, function parameters, and function return values. Marks may be deducted for too few type definitions, i.e. writing JavaScript instead of Typescript.
The movie data has the following fields:
 Name
 Director
 Year (i.e., release year)
 Genre (Comedy, Horror, Action, Drama, Thriller, Romantic)
 Notes (one line of comments that user can enter)
The movie data has the following requirements (1 mark):
• Each record must have a value for all fields, only the Notes field can be blank. There must be at least 10 records.
• The Genre must be one of the options shown above.
• A Name can only be entered and saved once (it must be unique for each movie).

CSC73010 – Programming Mobile Systems – Session 1 2020 Assignment 1
Your database would probably be a Typescript array of JavaScript objects, each of which has properties corresponding to the above fields. You can use another data structure if you wish but types will be necessary to ensure integrity of the data (1 mark).
Your app should implement:
• A way to edit the movie data (2 marks). Remember you do not have to permanently save the data between browser sessions, but you do have to save it while the app is active. HTML form widgets are the easiest here.
• A way to add a new movie data (2 marks). Consider re-using the edit form above, it will require careful thought so as not to confuse the user.
• A way to delete individual movie with a warning message saying “Do you really want to…” to reduce accidental deletions (2 marks).
• A way to search any movie by the Name field (2 marks). This will probably done with a HTML list but you may use other techniques (e.g. implement a search text field).
Your app should also:
• Provide an interface equally viewable by narrow and wider mobile screens (it will be tested). Note that this is just a request to keep your app simple and displayable on narrow screen as well as possibly wider screens. You can put links in a long page to navigate your app on narrow screens. This will be important when we get to mobile device apps later in the unit (1 mark).
• Use interactive features (e.g. use innerHTML assignments) to improve user experience (not alert() calls!). You are expected to add error messages where necessary (1 mark).
Part 2 – Expand the App (13 marks)
Modify the app from Part 1 to become a multipage Angular app. For example, place the add record on a separate page to the search facility. You can use as many pages as you like but don’t forget to use an error page (for malformed URLs), an Information page e.g., about the app and have your name as app developer. A Help page showing how to use the app would be nice as well.
Marks are allocated as follows:
• Having a working Angular app with appropriate Angular coding (3 marks). Minimally the app will have one component and one module, but to implement the following requirements you will need more components.
• Using Angular forms (3 marks). This is separate to the multipage facility. If you just add Angular code to the one-page Typescript app from Part 1, you have not used Angular forms.
• The multipage facility using the Angular router (3 marks). This includes the navigation between pages, error indicators and no broken links.
• Add at least one help button on each page that shows appropriate help messages. You should make interaction with the help facility as easy as possible. Also add helpful information, including error messages, when data entry is incorrect (3 marks).
• Use of CSS styles (1 mark). Just use them, there are no marks for artistic design. We are looking for the ability to use styles. Feel free to use styles you find in the Angular documentation or

CSC73010 – Programming Mobile Systems – Session 1 2020 Assignment 1
other on-line sources (a reference to the original designer should be made in your code if appropriate).
WARNING! Plagiarism is an academic misconduct and would be reported to the misconduct committee in this unit. Your program must be developed by yourself and should be original (not copied from web or friend). Please find details on plagiarism and other academic misconducts in this link: https://www.scu.edu.au/media/scueduau/staff/teaching-and-learning/ctl-document-downloads/as- academic-integrity-guides/Student-Academic-Misconduct-an-Introduction.pdf.

COMP2212 Programming Language Concepts Coursework – Submission Two |Haskell|平时作业|

COMP2212 Programming Language Concepts Coursework – Submission Two

Julian Rathke Semester 2 2019/20

Second Five Problems

Here are the second set of problems that I would like you to solve in your new Stream Programming Language. The input and output conventions are exactly as before but I repeat them here for convenience.

Input and output conventions

We will model streams using sequences of integers and, in general, inputs will take the following form:

a11 a21 a31 a41 … an1 a12 a22 a32 a42 … an2 …
a1k a2k a3k a4k … ank …

where every line is terminated with the new line character EOL and the file ends with EOF. The as are arbitrary (positive or negative) 32 bit integers (you can safely use a 64 bit implementation, we will not test for architecture specific overflow behaviour). The number n is the number of input sequences, which are the columns in the input. You can assume that all of the input streams will have the same length, or in other words, that all of the columns will be of the same size. This length will be finite in all tests but may be arbitrarily large. Note that empty inputs are allowed: if you are expecting n sequences and are given an empty input, you should assume that your input is n empty sequences. The values in each row are separated by a single space character. For example, the following is an input that describes 2 sequences of length 3:

21 3 −1 41

For each problem you are expected to produce a single output on stdout. Your output is expected be of the form:

o1
o2
o3 … oN

where the length of the output—that is, the number of lines—will depend on the length of the input. Where the output depends on an input stream that has exhausted its input then no further outputs are required. Every line must be terminated with the Unix new line character EOL (i.e. not Windows CR+LF).

1

Problem 6 – Two-Three Shuffle

Take two sequences a1 a2 a3 a4 a5 … and b1 b2 b3 b4 b5 b6 … as an input and output the sequence a1 a2 b1 b2 b3 a3 a4 b4 b5 b6 . . ., that is, the output sequence alternates between in- put sequences, but reads twice from input one and then three times from input two.

Example input: Expected output: 1

2 155 266 377 483 4 8

Note that input value 8 in the second input stream is consumed as this is the next logical output and is available in the (second) input stream.

Problem 7 – Skip Two then Three

Take a sequence a1 a2 a3 a4 a5 a6 . . . and produce the sequence a3 a7 a10 a14 a17 a21 …

This is the values from the input stream where we skip over two elements, output the next, then skip over three elements, output the next. This is then repeated for the next entries in the input streams.

Example input: 0
0
1

Expected output:

0
0
0 21 02 03 34 05 0

0 4 0 0 5

Problem 8 – Checksum Differences

Take two sequences a1 a2 a3 a4 a5 … and b1 b2 b3 b4 b5 … , and produce the sequence
a1 b1 a2 b2 a3 b3 a4 b4 a5 b5 (a1 −b1 +a2 −b2 +a3 −b3 +a4 −b4 +a5 −b5)…

That is a shuffle of the first input stream and the second input stream but after taking 5 entries from each of the input stream, a checksum value consisting of the the sum of the last five entries of the first input stream less the sum of the last five entries of the second input stream.

2

Example input:

15 24 33 42 51 60

Problem 9 – Counter Padding

Expected output: 1
5
2

4 3 3 4 2 5 1 0 6 0

Take a sequence a1 a2 a3 a4 a5 a6 . . . and produce the sequence
a1 1 a2 2 a3 3 a4 4 …an n…

That is the single input stream is copied to output but in between each entry is a incrementing counter value.

Example input:

23 12 53 90 27 4

Problem 10 – Fibonacci Sequences

Expected output: 23
1
12

2 53 3 90 4 27 5 4 6

Take a sequence a1 a2 a3 a4 a5 . . . and output the sequence
a1 a1 +a2 2a1 +a2 +a3 3a1 +2a2 +a3 +a4 5a1 +3a2 +2a3 +a4 +a5 …

where the coefficients of each input term in the sums follows the Fibonacci series 1 1 2 3 5 8 . . . from when it first appears. Recall that the Fibonacci series starts with two 1s and then the subsequent terms are always the sum of the previous two.

Example input: Expected output: Example input: Expected output: 1111 0123 0237

0 3 4 14 0 5 5 26

3

Second submission – due Thursday May 14th 4pm
What to submit. You will need to submit a single zip file containing:

First, provide the source code for five programs named (pr6.spl, pr7.spl, pr8.spl, pr9.spl, pr10.spl) written in your language that solve the additional problems. We will run our tests on your solutions and award marks for solving the additional problems correctly.

Second, a 3 page report on your language in pdf format named ‘report.pdf’ that explains the main language features, its syntax, including any scoping and lexical rules as well as addi- tional features such as syntax sugar for programmer convenience, type checking, informative error messages, etc. In addition, the report should explain the execution model for the interpreter, e.g. what the states of the runtime are comprised of and how they are transformed during execution. This report, together with the five programs will be evaluated qualitatively and your marks will be awarded for the elegance and flexibility of your solution and the clarity of the report.

Third, a text file called ‘group.txt’ that contains, the usernames of all students in your pair or group, as well as an agreed declaration of how marks are to be distributed amongst the members of your group. For example, 50-50, 60-40 etc. If such an agreed declaration cannot be made then please state this in your declaration instead and arrange for each member of the group to submit separately. Otherwise, please make sure that there is only one submission per group.

How to submit. All submissions must be done through handin.

Marks. This coursework counts for 40% of the total assessment for COMP2212. There are a total of 40 marks available. These are distributed between the two submissions as follows:

Submission one has 10 marks available. There are 2 marks available for functional correctness of each of the first five problems. You will receive the results of Submission 1 prior to the second deadline.

Submission two has 30 marks available. We will award up to 10 marks for the qualitative aspects of your solution, as described in your programming language report. We will award up to 20 marks for your solutions to the second five problems. For each problem there will be 4 marks available for functional correctness only. You have the option of resubmitting the interpreter, for a 50% penalty on this component. If you decide to resubmit your interpreter in the second submission the maximum possible total coursework mark is therefore capped at 30 marks.

Any late submission to either component will be treated as a late submission overall and will be subject to the standard university penalty of 10% per working day late.

4

COMS3200/7201 Assignment 2 |平时作业|network|python

COMS3200/7201 Assignment 2

Due: 25th May 2020, 20:00
100 marks total (20% weight for the final grade (100%))

Part A (30 marks total)

You are asked some questions on packet behaviours in the following network, which depicts three LANs (LAN1, LAN2 and LAN3) and a file server (S) connected to the internet.

Answereachofthefollowingquestionsintheassociatedquizonblackboard,followingthespecifiedinstructions. Each student will receive different network details.

  • All questions will be automatically marked.
  • Each question follows on from the previous question – i.e.: for each question you can assume the tasks described in the previous question(s) have been fully completed.
  • In any column that asks you for a protocol, the following options will be allowed: HTTP, FTP, Secure Shell (SSH), DNS, DHCP, UDP, TCP, ICMP, SNMP, ARP, OSPF, BGP.
  • In any case where multiple protocols are used, pick the highest-layer protocol (e.g., if SSH was used, you would select it over TCP, even though SSH packets use TCP as a transport layer protocol).
  • In any column that asks you for MAC address use upper case, two-digit hex format (e.g., AB:CD:EF:01:10:20 09:08:07:AA:BB:CC)
  • In any column that asks you for IP address use the dotted decimal format (e.g. 1.2.3.4 255.255.0.0).
  • Assume and only use 255.255.255.255 as the broadcast IP address of all the LANs depicted in this network.
  • Assume that the broadcast MAC address of all the LANs is FF:FF:FF:FF:FF:FF.
  • Assume all forwarding tables in the switches and routers are up to date.

1

Answer the following questions based on the network diagram and customised questions shown to you on Blackboard. Each student’s diagram and question values will be different for each submission.

1. Suppose Host A has just joined the network and does not currently have an IP address. Complete the following table describing the four packets that will be transmitted through the network upon this event (in the order they were sent), assuming that all requests succeed, and that Host A ends up with the IP address 4001 (i.e. customised information will be given here.)

Table 1.
Protocol Opcode Src MAC addr Dst MAC addr Src/Sender IPv4 addr Dst/Target IPv4 addr

2. Now suppose the Host A wants to send a standard DNS query to D2. The Host A has already gotten the IPv4 addresses of D2 and its gateway router from D1. R1’s ARP table contains the IP and MAC addresses of all hosts in the network, and all other ARP tables are empty. Complete the following table describing the first four packets that will be transmitted through the network upon this event, assuming that there are no other ongoing communications.

Table 2.
Protocol Opcode Src MAC addr Dst MAC addr Src/Sender IPv4 addr Dst/Target IPv4 addr

3. The Host A now wants to establish a HTTP connection with a server (S) (file server). You may assume that the Host A knows S’s IP address and that all ARP tables contain all required information to transmit any packet. Complete the following table describing the packets that are sent to transmit Host A’s request to initialise the connection, assuming the request is being sent from TCP port 4001 and the next available port number in R1’s NAT table is 5001. Give the TCP flags as an 8-bit binary number representing the CWR to FIN flags.

Table 3.
Protocol TCP Flags Src TCP port Dst TCP port Src/Sender IPv4 addr Dst/Target IPv4 addr

TCP TCP

2

  1. Suppose in addition to the connection described in (3), the Host B’s port 3200 is also currently connected to S. What are the contents of the NAT table in R1 while these connections remain alive? (if a new port is required, use port). Table 4.
    LAN IP LAN Port WAN IP WAN Port
  2. Suppose sent a packet with destination IP address 255.255.255.0 and destination MAC address What network devices (hosts, servers, routers, switches) in the above diagram would receive this packet?
  3. Suppose S sends a packet with the destination IP address 192.168.1.2. What network devices (hosts, servers, routers, switches) in the above diagram would receive this packet?

Part B (70 marks)

In this part of the assignment you will be required to implement the network layer for a host running on a virtual IP network. You should be familiar with the typical TCP/IP stack, where Ethernet is used as a link-layer protocol and IPv4 used as a network layer protocol. The protocols we will be using will be virtual – that is, network layer addresses won’t actually correspond to physical interfaces. To do this, the network stack will be redefined to that shown in the table below.

Host B

FF:FF:FF:FF:FF:FF.

TCP/IP Stack

Protocols

Virtual Stack

Network Layer

Link layer

Transport layer UDP Network layer IPv4

Link Layer

Ethernet

Figure 1.

As described in Figure 1, UDP will be used as your virtual network’s link layer, and you will be required to implement a virtualisation of IPv4 on top of this UDP-based link layer. Each UDP socket will correspond to a link layer interface, and localhost UDP port numbers will be used as the link layer addressing system of this virtual network.

By the end of this assignment, your implemented host program should be able to: • Accept simple user commands through a basic command line interface (CLI)

In other words, your program is the simulation of a host on a virtual network running on your computer. UDP port

numbers will be used as “MAC addresses”. Your program should only connect to localhost, i.e. all actual packets

should transmit across localhost.

3

• Sendandreceivemessagesacrossthisvirtualnetworklayer • HandlefragmentationofvirtualIPpackets

Program Invocation

Your program should be able to be invoked from a UNIX command line as follows. It is expected that any Python programs can run with version 3.6, and any Java programs can run with version 8. The IP-addr and LL-addr parameters correspond to the IPv4 address in CIDR notation (indicating the client’s subnet) and link layer address (UDP port number) of your host program respectively.

Python python3 NetworkSim.py IP-addr LL-addr C/C++ make

./NetworkSim IP-addr LL-addr

Java make
java NetworkSim IP-addr LL-addr

IMPORTANT: As the assignment is auto marked, it is important that the filenames and command line syntax exactly matches the specification. Specification adherence is critical for passing.

Your Task
1.1 Command Line Interface (5 marks)

To start with, you should implement a basic command line interface that will allow the user to supply basic information about the network. Your CLI should prompt users with a single > character, followed by a space. For the rest of the assignment we define anything wrapped in square brackets as a parameter or field that needs replacing (NOT as an optional parameter). The CLI should persistently prompt the user for another command until the program is terminated. For full marks in this section your CLI needs to accommodate the following commands:

  • gw set [IP-addr]: set the gateway IP address of the subnet the client is a part of to [IP-addr] (overriding any existing gateway address)
  • gw get: print the currently stored gateway IP address to stdout, or None if no gateway address has been specified
  • arp set [IP-addr] [LL-addr]: insert a mapping from [IP-addr] to [LL-addr] in the host’s ARP table (overriding any existing entries for [IP-addr])
  • arp get [IP-addr]: print the currently stored link layer address mapped to [IP-addr] to stdout, or None if no mapping exists
  • exit: terminate the program Error-handling of user input is optional – you won’t receive any marks for this, but you can choose to implement it provided it doesn’t impact on the rest of the assignment.

4

1.2 Sending Messages (15 marks)

To receive marks in this section, your CLI should be able to handle the following additional command:

• msg [IP-addr] “[payload]”: send a virtual IPv4 packet to [IP-addr] with the given payload (which will be supplied as a string)

Any packets sent should have a meaningful identifier and a protocol number of 0. You may assume the that all payloads will be less than or equal to the MTU of the network’s links, but the DF flag should still be set to 0. Your program should support any IP address, regardless of whether it is a part of your subnet or not.

If your program needs to send a packet to the gateway address and no gateway address has been specified, your program should print No gateway found to stdout. If no required address mapping can be found in the host’s ARP table, your program should print No ARP entry found to stdout.

1.3 Receiving Messages (15 marks)

In addition to sending packets, your program should be able to receive them from other hosts. When your program receives an IPv4 packet with the protocol indicator set to 0, it should print the payload of that packet to stdout, in the below format ([IP-addr] should be replaced with the sender’s IPv4 address and [message] should be replaced with the string encoding of the payload):

Message received from [IP-addr]: “[message]”

Note that there is only a single space after the colon. When your program receives an IPv4 packet with a non-zero protocol number, it should print the following message to stdout ([proto-num] should be the hexadecimal representation of the protocol formatted as 0x??):

Message received from [IP-addr] with protocol [proto-num]

You can assume all packets sent to your program are valid IPv4 packets. You should be able to receive messages at any time without blocking the CLI, and any messages should be printed cleanly (without any CLI prompts or responses disrupting the message contents). Remember to backspace the current prompt (the > character) before printing the output.

1.4 IP Fragmentation (15 marks)

To receive marks in this section your program should be able to handle IP fragmentation. Your CLI should support the following extra commands:

• mtu set [value]: set the MTU of the network’s links as the specified [value]

• mtu get: print the currently stored MTU (the default MTU should be 1500)

Virtual packets that are longer than the specified MTU (or the default MTU if none has been specified) should be fragmented before transmission. The length of a virtual packet is equal to the length of the IPv4 header added to the length of the IPv4 payload (i.e. you don’t need to consider the length of the non-virtual headers). You can assume the value of the MTU will never be smaller than 100.

Your program should also be able to receive packets that have been fragmented (and display them as a single message).

the client’s IP in CIDR notation. When a packet is sent to an IP address outside of the subnet range, it should be sent to

the gateway address.

The subnet mask can be extracted from

Hint: The escape character for backspace is “\b”. For example, use print(‘\b’) to backspace a previously printed

character in Python.

5

1.5 Handling packet loss (20 marks)

To receive marks in this section your program should be able to handle packet losses. Your CLI should support the following extra commands:

• frto set [value]: set the FRTO ( Fragment Reassembly TimeOut) of the network to the specified [value] • frto get: print the currently stored FRTO (the default FRTO should be 5)

When fragmented packets are received these should be held in memory until all fragments for the given identification field (Id) have arrived, or until FRTO expires. Each Id should have its own FRTO timer, which starts with the arrival of a new packet Id. Once the FRTO expires, all fragments related to the FRTO should be discarded and a packet with a protocol number of 114 and the identification field set to Id should be returned to the sender.

When sending messages, you should keep a collection of fragmented packets for a maximum of 3 x FRTO (i.e. 3 times FRTO time), and resend the fragments related to the Id when you get a packet with the protocol number 114. Changing the FRTO value (using frto set) does not affect any existing timers in operation.

While printing non-zero protocol numbers remains unchanged, when the protocol number is 114, and the fragments being requested for are available (valid and not expired), your program should print the following message to stdout instead. ([fragmentation-Id] should be the hexadecimal representation of the identification field as 0x??)

Message-Resend received from [IP-addr] for id [fragmentation-Id]

Fragments older than 3 x FRTO are expired and should be deleted. If you get a packet with protocol 114 for an Id that is expired or unknown, these should be ignored, and the following message printed to stdout.

Message-Resend received from [IP-addr] for id [fragmentation-Id] BAD

6

Example CLI output

python3 NetworkSim.py 192.168.1.1/24 1024

> gw get

None

None
2222
1500
1600

5 3

Packet Formatting

Your IPv4 packets don’t need to contain meaningful values for the differentiated services code point (DSCP)/Explicit Congestion Notification (ECN) fields, nor are you required to compute a correct checksum. You can assume packets will never contain any options and subsequently that the Internet Header Length (IHL) will always be 5. The TTL field should contain some meaningful value (i.e. to allow the packet to reach its destination).

Tips for Success

  • RevisitthelecturesandlabsontheinternetlayeroftheTCP/IPstack(inparticularthelecturesonIPv4).
  • Make sure you fully understand the requirements of this assignment before commencing work.
  • Ensure you always exercise good thread safety if using threads to implement concurrency.
  • Frequently test your code on MOSS.
  • Ensure your base functionality is working (such as the CLI) before attempting the more difficult tasks.
  • Start early and ask any questions you might have early.

> gw set 192.168.1.30

> gw get

192.168.1.30

> msg 192.168.1.2 “hello”

No ARP entry found

> arp get 192.168.1.2

> arp set 192.168.1.2 2222

> arp get 192.168.1.2

> msg 192.168.1.2 “hello”

> mtu get

> mtu set 1600

> mtu get

Message received from 192.168.1.2: “hello there, thank you for your message”

Message received from 192.168.1.3 with protocol 0x06

> frto get

> frto set 3

> frto get

Message-Resend received from 192.168.1.6 for id 0x00a4

Message-Resend received from 192.168.1.7 for id 0x00a5 BAD

> exit

7

Library Restrictions

• The only communication libraries you may use are standard socket libraries which open UDP sockets.

• You can’t use any libraries that aren’t considered standard for your language (i.e. if you have to download a library to use it would be considered as non-standard).

• If you are unsure about whether you may use a certain library, please ask the course staff on Piazza.

Submission

Submit all files necessary to run your program. At a minimum, you should submit a file named NetworkSim.py, NetworkSim.c, NetworkSim.cpp or NetworkSim.java. If you submit a C/C++ or Java program, you should also submit a makefile to compile your code into a binary named NetworkSim or a .class file named NetworkSim.class.

IMPORTANT: If you do not adhere to this (e.g. submitting a C/C++/Java program without a Makefile, or a .class file instead of a .java file), you will receive 0 for this part of the assignment. Do not submit executables.

Marking

Your code will be automatically marked on a UNIX machine, so it is essential that your program’s behaviour is exactly as specified above. Your program should complete all tasks within a reasonable time frame (for example a single packet should not take more than one second to construct and send) – there will be timeouts on all tests and it is your responsibility to make sure your code is not overly inefficient. It is expected that you will receive a small sample of tests and a basic RUSH2 client program before the submission deadline.

There are no marks for coding style, however you may lose marks in relevant sections for poor management of concurrency.

Academic Misconduct

Students are reminded of the University’s policy on student misconduct, including plagiarism. See the course profile and the School web page http://www.itee.uq.edu.au/itee-student-misconduct-including-plagiarism.

Late Submission and Extensions

Please view the ECP for policies on late submissions and extensions as well as updates based on COVID-19 situation.

Version History

12/05/2020 Initial release

1.0

8

COMP2212 Programming Language Concepts Coursework |Haskell|代码代写

COMP2212 Programming Language Concepts Coursework

Julian Rathke Semester 2 2019/20

Introduction

Streams are potentially unbounded sequences of data. They are common in mathematics, com- puter science and everyday life. Examples of streams include streaming a movie on the web, a sequence of bits flowing through a wire in a digital circuit, or temperature measurements outside of Building 53 taken every hour. The theory of (discrete) signal processing deals with infinite sequences of reals. Similarly, in computer science, dataflow is the study of networks of nodes and channels through which a potentially unbounded number of data elements flow. A particular feature of streamed data is that streams may have different rates of accessing data associated with them.

Your task is to design and implement a domain specific programming language that performs certain simple computations on potentially unbounded integer sequences. You are required to

1. Invent an appropriate syntax,
2. Write an interpreter for your language using Haskell, and 3. Document your language in a written report.

The coursework will consist of two submissions. More details, including advice and deadlines can be found at the end of this document.

Submission 1

For the five example problems listed below, you will be required to produce a program, in your language, that solves each of the problems. Therefore, your design should be guided by the five problems, since you are required to produce source files in your own language that solve them.

The first submission consists of the source for your interpreter together with solutions written in your language to the first five problems.

Submission 2

Please keep a record of all the sources you consult that influence your design, and include them in your programming language report. The report will be required for the second submission, along with programs for five additional unseen problems, which we will make public after the deadline for the first submission.

The five additional problems will not ask you to reinvent the wheel. Instead, they will consist of variations and combinations of the kinds of computations that appear in the first five problems. The idea is to test the orthogonality of your design.

The specification is deliberately loose. If we haven’t specified something, it is a design decision for you: e.g. how to handle syntax errors, illegal inputs, whether to allow comments, support for syntax highlighting, compile time warnings, any type systems etc. A significant part of the mark (25%) will be awarded for these qualitative aspects, where we will also take into account the design of the syntax and reward particularly creative, clean and concise solutions with additional marks. The remaining 75% of the mark will be awarded for solving the problems correctly.

1

First Five Problems Input and output conventions

We will model streams using sequences of integers and, in general, inputs will take the following form:

a11 a21 a31 a41 … an1 a12 a22 a32 a42 … an2 .
.

.

a1k a2k a3k a4k … ank .

.

.

where every line is terminated with the new line character EOL and the file ends with EOF. The as are arbitrary (positive or negative) 32 bit integers (you can safely use a 64 bit implementation, we will not test for architecture specific overflow behaviour). The number n is the number of input sequences, which are the columns in the input. You can assume that all of the input streams will have the same length, or in other words, that all of the columns will be of the same size. This length will be finite in all tests but may be arbitrarily large. Note that empty inputs are allowed: if you are expecting n sequences and are given an empty input, you should assume that your input is n empty sequences.

The values in each row are separated by a single space character. For example, the following is an input that describes 2 sequences of length 3:

21 3 −1 41

For each problem you are expected to produce a single output on stdout. Your output is expected be of the form:

o1 o2 o3 .

.

.

oN

where the length of the output—that is, the number of lines—will depend on the length of the input. Where the output depends on an input stream that has exhausted its input then no further outputs are required. Every line must be terminated with the Unix new line character EOL (i.e. not Windows CR+LF).

2

Problem 1 – Double Speed Shuffle

Take two sequences a1 a2 a3 a4 a5 … and b1 b2 b3 b4 … as an input and output the sequence a1 a2 b1 a3 a4 b2 …, that is, the output sequence alternates between input sequences, but reads twice from input one for every read from input two.

Example input: Expected output: 1

152 265 373 484

6

Note that input values 7,8 in the second input stream are never consumed as the first input stream has been exhausted.

Problem 2 – Shuffling and Arithmetic

Takethreesequencesa1 a2 a3 a4 a5 …,b1 b2 b3 b4 b5 …andc1 c2 c3 c4 c5 …asinputsand output the following sequence

c1 b1 a1 a1 +b1 b1 +c1 c2 b2 a2 a2 +b2 b2 +c2 …

This is the values from input stream three, then two then one, followed by the sum of the same values read from streams one and two and finally the sum of the values read from streams two and three. This is then repeated for the next entries in the input streams.

Expected output: 8
2
1

3 10

−4 128 7 0 7 −4 0 3 5 −2 7 3

−2 5 3 8 3

Problem 3 – Skip and Prefix

Take two sequences a1 a2 a3 a4 … and b1 b2 b3 b4 … , and produce the sequence 0 b2 a1 b4 a2 b6 a3 b8 …

That is a shuffle of the first input stream prefixed by a fixed value 0, and the second input stream where every other value is skipped over.

Example input:

3

Example input:

15 24 33 42 51 60

Expected output: 0
4
1

2 2 0 3

Problem 4 – More Stream Arithmetic and Local Reverse

Take a sequence a1 a2 a3 a4 a5 a6 . . . and produce the sequence
a3 2∗a2 3∗a1 −1 a6 2∗a5 3∗a4 −1 …

That is at least three values are taken from the single input sequence, the third of these is output first, the second is doubled and then output, finally the first value is trebled, decremented and output. This sequence repeats.

Example input: Expected output: 13 24 32 46

5 10 6 11

Problem 5 – Accumulator

Take a sequence a1 a2 a3 a4 . . . and output the sequence
a1 a1 +a2 a1 +a2 +a3 a1 +a2 +a3 +a4 …

where each term of the output is the sum of all the input terms up to that point.

Example input: Expected output: 11 23 36

4 10

4

First submission – due Monday April 27 4pm Interpreter and the first five problems

You are required to submit a zip file containing:

• the source for the interpreter for your language, written in Haskell

• five programs, written in YOUR language, that solve the five problems specified above. The programs should be in files named pr1.spl, pr2.spl, pr3.spl, pr4.spl, pr5.spl.

Implementation and compilation. We will compile by using the command make, so you will need to include a Makefile in your zip file that produces an executable named myinterpreter. Prior to submission, you are required to make sure that your interpreter compiles on a Unix machine with a standard installation of GHC (Version 8.4.3) or earlier: if your code does not compile then you will be awarded 0 marks. Before submission, we will release a simple “automarking” script that will allow you to check if your code compiles and whether each of your programs passes a basic test.

You can use Alex and Happy for lexing and parsing. Alternatively you can use any other Haskell compiler construction tools such as parser combinator libraries. You are welcome to use any other Haskell libraries, as long as this is clearly acknowledged and the external code is bundled with your submission, so that it can compile on a vanilla Haskell installation.

Interpreter spec. Your interpreter is to take a file name (the program in your language) as a single command line argument. The interpreter should expect its input on standard input (stdin), produce output on standard output (stdout) and error messages on standard error (stderr).

For each problem, we will test whether your code performs correctly by using a number of tests. We only care about correctness and performance will not be assessed (within reason – our marking scripts will timeout after a generous period of time). You can assume that for the tests we will use correctly formatted input. For example, when assessing your solution for Problem 1 we will run

./myinterpreter pr1.spl < input.txt
where input.txt will contain the input streams for a particular test, following the input conven-

tions described earlier.

Second submission – due Thursday May 7th 4pm Written Report and the second five problems

Shortly after the first deadline we will release a further five problems. Although they will be different from Problems 1-5, you can assume that they will be closely related, and follow the same input/output conventions. You will be required to submit two separate files.

First, you will need to submit a zip file containing programs (pr6.spl, pr7.spl, pr8.spl, pr9.spl, pr10.spl) written in your language that solve the additional problems. We will run our tests on your solutions and award marks for solving the additional problems correctly.

Second, you will be required to submit a 3 page report on your language in pdf format that explains the main language features, its syntax, including any scoping and lexical rules as well as additional features such as syntax sugar for programmer convenience, type checking, informative error messages, etc. In addition, the report should explain the execution model for the interpreter, e.g. what the states of the runtime are comprised of and how they are transformed during execution. This report, together with the five programs will be evaluated qualitatively and your marks will be awarded for the elegance and flexibility of your solution and the clarity of the report.

Please note: there is only a short period between the first and second submission. I strongly advise preparing the report well in advance throughout the development of the coursework.

5

The coursework is to be done in pairs. As part of the second submission we will require a declaration of how marks are to be distributed amongst the members of your group. All submis- sions will be done through handin. Please make sure that there is only one submission per group – there will be a marks penalty if both members submit separately.

Marks. This coursework counts for 40% of the total assessment for COMP2212. There are a total of 40 marks available. These are distributed between the two submissions as follows:

Submission one has 10 marks available. There are 2 marks available for functional correctness of each of the first five problems. You will receive the results of Submission 1 prior to the second deadline.

Submission two has 30 marks available. We will award up to 10 marks for the qualitative aspects of your solution, as described in your programming language report. We will award up to 20 marks for your solutions to the second five problems. For each problem there will be 4 marks available for functional correctness only. You have the option of resubmitting the interpreter, for a 50% penalty on this component. If you decide to resubmit your interpreter in the second submission the maximum possible total coursework mark is therefore capped at 30 marks.

Any late submission to either component will be treated as a late submission overall and will be subject to the standard university penalty of 10% per working day late.

6

SWIFT|IOS|ASSIGNMENT|PROJECT|GUI

Project 2 Description

Faculty of Engineering and Information Technology School of Computer Science

41889 – Application Development in the iOS Environment 42889 – iOS Application Development
Autumn 2020

Assessment Task 2

Individual programming project: BubblePop Game
Due 22 May 2020 at 11:59 pm
This project is worth 30% of the overall mark for this subject. GitHub Classroom invitation

Introduction

A client has just hired you to develop a casual iOS game based on an idea he had recently. He has no programming experiences and only knows the overall functions the game must have. You must help him to work out a detailed Graphical User Interface (GUI), design a well- structured program and implement the program in the Swift programming language. The final product shall be thoroughly tested and ready to be submitted to the App Store.

Figure 1. Gameplay Mockup Figure 2. Scoreboard Mockup

Objectives

The purpose of this project is to demonstrate competence in the following skills:

The full software development cycle iOS App UI and program design
iOS UI programming
Effective use of platform APIs Program testing

Functionality Specification

The game is called BubblePop (you can call your app a different name if you wish to). In this game, a number of bubbles are randomly displayed on an iOS device screen. A player pops a bubble by touching the bubble, and every time they pop a bubble they earn a certain number of points. Bubbles come in five colours: red, pink, green, blue and black. Each colour corresponds to a specific number of points and has a specific probability of appearance (see table 1). All bubbles appear on the screen briefly (see core functionality 9). A player needs to pop as many bubbles as possible within a certain timeframe (default to 60 seconds) to get high scores. Note that, if a player pops two or more bubbles of the same colour consecutively, they earn 1.5 times the points for the additional bubbles they pop. Finally, game scores are saved and a high score board is displayed after a game run is finished. Figure 1 provides a rough mock up of what the main interface may look like. NOTE: Figure 1 gives you a guide for your GUI, you may design a

more sophisticated and more polished GUI for your own work. You will get 1 functionality mark for each functionality item specified below.

Colour

Red

Pink

Green

Blue

Black

Game Points

1

2

5

8

10

Probability of Appearance

40%

30%

15%

10%

5%

Table 1. Specifications for coloured bubbles

Core Functionalities

  1. A player must be able to enter their name before the start of a game run. (You may load the player’s name from the GameKit API if you wish. This extra work is not required.)
  2. A game timer shall be displayed and it must count down continuously in one-second intervals. When the timer reaches zero, the game stops.
  3. A score shall be displayed. It shall be zero initially and shall be updated every time the player successfully “pops” a bubble.
  4. The default timeframe for a game is 60 seconds, i.e. the game timer starts from 60 seconds and counts down to 0. This number shall be adjustable in the app settings.
  5. The maximum number of bubbles displayed on the screen is defaulted to 15, i.e. there shall be between 0 and 15 bubbles shown on the screen at the same time. This number shall be adjustable in the app settings.
  6. The app randomly decides how many bubbles (up to the maximum allowed bubble number) shall be displayed on the screen at a time. The bubble colour is decided according to the Probability of Appearance in Table 1.
  7. Bubbles shall be displayed at random positions on the screen with the following restrictions: 1. The entire bubble shall be displayed within the screen. There shall not be any bubble with some parts off the screen once it has fully appeared. 2. No two (or more) bubbles shall overlap each other.
  8. When a player touches a bubble, the bubble shall be removed from the screen and the corresponding game points shall be added to the overall score. For example, if the green bubble in Figure 1 was touched, it shall disappear and the score shall increase by 5 game points. If two or more bubbles of the same colour are popped in a consecutive sequence, the bubbles after the first one will get 1.5 times their original game points. For example, if two black bubbles were popped one after the other, 25 (10 + 1.5 * 10) shall be added to the total score. Round to the nearest integer if necessary.
  1. The app shall refresh bubbles displayed every game second. That is, after every game second, the app shall randomly remove a number of bubbles (do not include the bubbles that have been popped by the player) and replace them with another set of randomly positioned bubbles. There may be more or less bubbles on the screen compared with the previous game second subjected to the restrictions in 5 and 6. In this case, “random” means chosen by the program, not the user. So you have a lot of discretion in placement of new bubbles and selection of old bubbles.
  2. Whenthegamestops,theplayer’sscoreshallbesavedinapersistentfile(ordatabase) that contains all players’ names and their highest scores.
  3. Attheendofthegame,ahighscoreboardshallbedisplayedwiththenamesandscores of the highest ranking players. (See Figure 2)

Extended Functionalities (please do not start on these until finishing CFs)

  1. In addition to core function 7 and 9, displayed bubbles shall move and go off the screen if they are not removed earlier. Their moving speed shall increase as the game timer counts down. You decide the rate of change with respect to the game timer.
  2. Animations of game state transitions. For example, one of: 1. Flashing count down 3, 2, 1 start at the beginning of play
    2. Bubbles shrinking, growing, or flying away when removed or tapped
    3. Score changes or combo lengths displayed with animations in response to taps
  3. The highest score in the scoreboard shall be displayed during gameplay.
  4. Any cool and useful features that you can think of.

PROGRAM/TEST HINT

Make sure your program works under different iOS devices/simulators (e.g. iPhone4s, iPad etc.) with different screen sizes and orientations.
For core functionality 4 and 5, the game time and the maximum number of bubbles can be set in the app settings. How can you make sure their settings are valid? How can you prevent a user from setting a negative or a very large number for these settings. Think about how to communicate these constraints to the user in a comfortable way.

Assessment

You will demonstrate your app in your assigned lab on the week of 22 May. Your peers will mark your app’s functionality and give you feedback on usability.

Max score: 30 marks

Functionality: 20 marks (marked by peers)

The Xcode project must unzip successfully and compile without errors.

15 marks for specified functionalities.
Full marks in functionality requires the completion of all CFs and EFs

2 marks for compiling without warnings.

1 mark for operating without unhandled runtime errors.
2 marks for correct display on different screen sizes and orientations.

Code Style: 3 marks (marked by staff)

Deduct up to 1 mark for bad or inconsistent indentation, whitespace, or braces. Deduct up to 1 mark for bad or misleading comments.
Deduct up to 1 mark for unclear symbol naming.

Program Design: 7 marks (marked by staff)

Data modeling: Do the data structures reflect the problem domain?
Immutable data and idempotent methods: Is the type system used to prevent incorrect code from being written?
Functional separation: Is the problem broken down into meaningful parts?
Loose coupling: Can parts be changed in isolation of each other?
Extensibility: How easy would it be to add, remove, or change functionality? Could a new mechanic or new content be added by changing data instead of changing code?
Error handling: Are errors detected at appropriate places? Is the user prevented from entering invalid input and guided toward valid input?

Late submission

Late submissions may be demonstrated in lab the following week.

Deduct 1 mark per 24 hours late (rounded up). Submissions will not be accepted after the Final Due Date (7 days after the standard due date).

Please note: Regardless of how many times you have submitted your project, if your final submission is after the due date it will be considered late and marks will deducted accordingly.

An extension will only be granted if there is a fully documented reason which merits it. The documentation must be presented to the Subject Coordinator before the due date. Extensions after the Final Due Date will never be granted under any circumstance. If an extension is granted that means submission will be accepted up to the extension date without penalty. If an extension is granted, UTS Online will show the extended due date.

Students may apply for special consideration if they consider that illness or misadventure has adversely affected their performance.

Bug reports

It is quite possible that errors or ambiguities may be found in the specification. If so, updates will be placed on UTSOnline and announcements made regarding the amendment. It is your responsibility to keep up to date on such amendments and ensure you are using the latest version of the Project Description.

If you discover an error or bug in the provided material, you will receive favorable marking. The following rules apply:

1. It must be a report on the currently posted version of the material.

2. It must be reported on the UTS Online discussion board to be accepted.
3. It must be a genuine bug. By genuine I mean it requires me to amend the material.
4. If a number of students post a report on the same bug, the first who posted will receive

the mark.

Return of Assessed Project

It is expected that marks will be made available one week after the final due date via UTSOnline. You will also be given a copy of the marking sheet showing a breakdown of the marks and feedback.

Acceptable Practice vs Academic Malpractice

Students should be aware that there is no group work within this subject. All work must be individual. However, it is considered acceptable practice to adapt code examples found in the lecture notes, labs and the text book for the assignment. Code adapted from any other source, particularly the Internet and other student assignments, will be considered academic malpractice. The point of the assignment is to demonstrate your understanding of the subject material covered. It’s not about being able to find solutions on the Internet.

You should also note that assignment submissions will be checked using software that detects similarities between students programs.

Participants are reminded of the principles laid down in the “Statement of Good Practice and Ethics in Informal Assessment” in the Faculty Handbook. Assignments in this subject should be your own original work. Any collaboration with another participant should be limited to those matters described in the “Acceptable Behaviour” section. Any infringement by a participant will be considered a breach of discipline and will be dealt with in accordance with the Rules and By-Laws the University. The Faculty penalty for proven misconduct of this nature is zero marks for the subject. For more information, see UTS Policy on Academic integrity, plagiarism and cheating

Queries

If you have a question, please contact the instructor as soon as possible.

It is easier to answer queries about specific code. Please upload your code to the GitHub Classroom repository so that I can read it instead of guessing what it might be.

If the answer to your questions can be found directly in any of the following:

subject outline
task specification
UTS Online discussion board

You will be directed to these locations rather than given a direct answer.

考试代做|PYTHON|AI|GA|PSO|ES|CNN

ALTERNATIVE EXAM [3 hours, 4 points]:  A feedforward neural network architecture is given by a sequence of numbers of nodes in successive layers, e.g., (9, 7, 11, 12, 5) describes a network for a 9-input 5-output problem, with 3 hidden layers containing 7, 11, and 12 nodes.  Implement a GA whose goal is to discover the “best” network architecture for a given problem, minimizing 

(1-accuracy) . (# hidden layers) .  max(1, max(# nodes in any hidden layer – # input nodes)).

For example, if accuracy is 90% for a (9,7,11,12,5) network, then this evaluates to ((1-0.9)(3)(12-9)) = 0.9, better than a (9,11,14,5) network with 80% accuracy, since ((1-0.8)(2)(14-9))=2.0. 

1.     Use PSO or ES to train any network to obtain the accuracy. 

2.     Report the results obtained on a non-trivial problem with high input dimensionality, i.e., where good results are not obtained easily using a simple network. 

3.   Summarize your results, discussing the usefulness of this approach.

SPARK|PYTHON |Big Data Management & Analysis|大数据作业代写|hadoop|mapreduce

BDM_Final

CUSP-GX-6002.001: Big Data Management & Analysis SPRING 2020

Final Challenge – Parking Violations in NYC

OBJECTIVES:

In this challenge, we would like to gather statistics on the number of parking violations (tickets) per street segment in NYC over the past 5 years. In particular, for each street segment in NYC, we would like to have the following:

  1. The total number of parking violations for each year from 2015 to 2019.
  2. The rate that the total number of violations change over the years using Ordinary Least Squares.

You are asked to write a Spark program to compute the above metrics for all the street segments in NYC.

INPUT DATA:
Parking Violations Issued – Fiscal Year 2015-2019 (around 10GB in total)

Source: https://data.cityofnewyork.us/browse?q=%22Parking%20Violations%20Issued%22 Description: Datasetscontainparkingviolationsissuedduringtherespectivefiscalyear.Moreinfo (including data dictionary) can be found on one of the results listed in the above link. All 5 year data are available on HDFS, in their raw exported CSV format, under the following folder: hdfs:///tmp/bdm/nyc_parking_violations/

NYC Street Centerline (CSCL) (around 120k segments)

Source: https://data.cityofnewyork.us/City-Government/NYC-Street-Centerline-CSCL-/exjm-f27b Description: The NYC Street Centerline (CSCL) is a road-bed representation of New York City streets containing address ranges and other information such as traffic directions, road types, segment types. Details on this data set are available at: https://github.com/CityOfNewYork/nyc-geo- metadata/blob/master/Metadata/Metadata_StreetCenterline.md

The data is available in CSV format on HDFS as:

hdfs:///tmp/bdm/nyc_cscl.csv

METHODLOGY:

Generally, this challenge could be addressed by performing a spatial join of the violation data set with the street centerline one based on the issued location of each violation. However, there is no geospatial location (latitude and longitude) recorded for the violation data set. Instead, a street address is provided through the House Number; Street Name; and Violation County field. For example, if a violation was issued at “187 State St” in Brooklyn (aka. Kings county), values for the 3 fields would be:

House Number Street Name Violation County

187 State St K

Possibly, we could use a geocoding software (e.g. DOITT’s GeoSupport) or a geocoding web service (e.g. DOITT’s GeoClient) to help us converting addresses into geospatial locations. However, these approaches are

CUSP-GX-6002: Big Data Management & Analysis Page 1 CUSP

SPRING 2020

either restricted to a desktop environment or limited by the number of requests. In order to perform geocoding on a large amount of records such as 50 million parking violations, we will rely on the street centerline data set. The street centerline data set not only provides geometries of street segments but also records the house number range for each segment. For example, we should be able to find the following record in the data set:

PHYSICALID FULL_STREE BOROCODE L_LOW_HN L_HIGH_HN

58903 STATE ST 3 183 229

This means that the address “187 State St” in Brooklyn should be matched to the street segment with physical ID 58903. This is because the segment has the same street name (“STATE ST”, case-insensitive) and borough (“3” means Brooklyn), and the target house number (187) is within its range (183 to 229).

Note that in additional to the 2 “left” fields (L_LOW_HN and L_HIGH_HN) that captures the odd house number range, there are also R_LOW_HN and R_HIGH_HN corresponding to the even house number range. You have to select the right fields to perform the lookup depending on the house number (aka. odd or even). Also, house numbers may be expressed in a compound form such as 170-044, or 56-098. These should be treated as tuples (170,44) and (56,98). A valid range from the street centerline could be from (54,0) to (56,98). Not all street segments have house number.

For the parking violations data set, the Issue Date field should be used to determine which year a violation belongs to. The filename, which is organized by fiscal year, should not be used. We are only interested in those violations from 2015 to 2019. Violations outside of those years should not be counted.

Ordinary Least Squares: for simplicity, only the coefficient of your regression line (aka. the slope) is of interest. This is regardless of the estimated errors, residuals and/or coefficient of determination (aka. R-squared) of your regressor.

OUTPUT DATA:

Your output must be written to HDFS in CSV format (without header). It must contain the columns below, and the records must be sorted by PHYSICALID of street segments:

SAMPLE OUTPUT (header not to be included):

PHYSICALID,COUNT_2015,COUNT_2016,COUNT_2017,COUNT_2018,COUNT_2019,OLS_COEF

58903,1000,900,800,700,600,-100 …

where COUNT_XXXX is the total number of violations issued at the street segment PHYSICALID for year XXXX, and OLS_COEF is the OLS coefficient.

Note that all streets segments in the street centerline data set must be present in the output even when there are no violations issued (zero counts).

CUSP-GX-6002: Big Data Management & Analysis Page 2 CUSP

SPRING 2020

SUBMISSION:

Your code will be run with exactly 25 cores (5 executors and 5 cores per executor), and no more than 10GB of memory per executor. It must not take more than 30 minutes to complete.

SAMPLE RUN:
spark-submit –num-executors 5 –executor-cores 5 –executor-memory 10G \

–conf spark.executorEnv.LD_LIBRARY_PATH=$LD_LIBRARY_PATH \ BDM_FinalChallenge.py <OUTPUT_HDFS_FOLDER>

Note: the above command is only an example to demonstrate how to specify the number of executors. You may have additional files to be included with your submission, and must replace <OUTPUT_HDFS_FOLDER> with a real output location. There is no need to specify the input data through command line. The files are assumed to be located under /tmp/bdm/ as specified above. This means that you can hard-code the input filenames inside your code.

Please provide the following information when submitting your final challenge:

  • –  One or more files including your application Python file and any dependencies that it may need.
  • –  The spark-submit command needed to run your code.
  • –  The total time your code took to produce the results (when you last ran it successfully).
  • –  A CSV collected from your output folder (using hadoop fs -getmerge …) for sanity check.

CUSP-GX-6002: Big Data Management & Analysis Page 3 CUSP

SPRING 2020

Machine learning|Python|final project|AI

  1. Project PlanSelect a machine learning problem, and example data sets.Sketch an idea or approach to be explored.  You can choose a nature-inspired EA not explored in course assignments (ant, bat, bee, firefly, fish, …).Describe the criterion to be used for evaluation, and how evaluation will be carried out: you are expected to compare with GA/ES/PSO and a more traditional gradient-descent based learning algorithm.
  1. Final Project report and implementation code
    1. Report Format: IEEE 2-column 12-pt Times New Roman; with abstract and five sections: Introduction, Problem & Data Description, Approach, Results, Discussion.
    1. Code: Must demonstrate that it works.
    1. Evaluation results
    1. PPT to explain your work